diff --git a/README.md b/README.md index b3b43dd..feddb31 100755 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ This implementation uses the new features detailed in [this link](https://aws.am - Source generation - Executable assemblies -### Minimal API +### .NET6 Minimal API There is a single project named ApiBootstrap that contains all the start-up code and API endpoint mapping. The SAM template still deploys a separate function per API endpoint to negate concurrency issues. It uses the new minimal API hosting model as detailed [here](https://aws.amazon.com/blogs/compute/introducing-the-net-6-runtime-for-aws-lambda/). @@ -59,12 +59,6 @@ It uses the new minimal API hosting model as detailed [here](https://aws.amazon. Same as minimal API but instead of using Amazon.Lambda.AspNetCoreServer.Hosting/Amazon.Lambda.AspNetCoreServer it is based on [Aws Lambda Web Adapter](https://github.com/awslabs/aws-lambda-web-adapter) -### .NET 6 native AOT - -The code is compiled natively for either Linux-x86_64 or Linux-ARM64 and then deployed manually to Lambda as a zip file. The SAM deploy can still be used to stand up the API Gateway endpoints and DynamoDb table, but won't be able to deploy native AOT .NET Lambda functions yet. Packages need to be published from Linux, since cross-OS native compilation is not supported yet. - -Details for compiling .NET 6 native AOT can be found [here](https://github.com/dotnet/runtimelab/blob/feature/NativeAOT/docs/using-nativeaot/compiling.md) - ### .NET 7 Custom Runtime The code is compiled on a custom runtime and deployed to the provided.al2 Lambda runtime because Lambda doesn't have a .NET 7 runtime. The code is compiled as ReadyToRun and Self-Contained because there is not .NET runtime on provided.al2 to depend on. This type of deployment is expected to be slower than a fully supported Lambda runtime like .NET 6. This sample should be able to be tested with `sam build` and then `sam deploy --guided`. @@ -295,28 +289,6 @@ filter @type="REPORT" 36.71 111.28 - - Native AOT on ARM64 - 1277.19 - 1326.64 - 1358.84 - 1367.49 - 6.10 - 9.37 - 17.97 - 838.78 - - - Native AOT on X86 - 466.81 - 542.86 - 700.45 - 730.51 - 6.21 - 11.34 - 24.69 - 371.16 - ### .NET 7 @@ -557,6 +529,32 @@ The .NET 8 benchmarks include the number of cold and warm starts, alongside the 23.84 120.45 + + Minimal API on X86 + 195 + 1672.85 + 1737.61 + 1833.97 + 1889.15 + 136,006 + 6.30 + 10.98 + 26.72 + 124.67 + + + Minimal API on ARM64 + 242 + 1972.79 + 2049.16 + 2107.32 + 2124.55 + 136,816 + 6.01 + 9.37 + 24.69 + 331.6 + *The .NET 8 native AOT examples need to be compiled on Amazon Linux 2, this is a temporary solution as a pre-cursor to a SAM build image being available for .NET 8. diff --git a/loadtest/codebuild/load-test-buildspec.yml b/loadtest/codebuild/load-test-buildspec.yml index d2c1e0f..173cd0e 100644 --- a/loadtest/codebuild/load-test-buildspec.yml +++ b/loadtest/codebuild/load-test-buildspec.yml @@ -3,14 +3,23 @@ version: 0.2 phases: install: commands: - - npm i artillery -g + - cd loadtest/codebuild + - ./setup.sh build: commands: - - export temp1=$(pwd);echo $temp1 - - cd loadtest/codebuild - ./run-all-load-tests.sh artifacts: files: + - src/NET6/Report/* + - src/NET6Containers/Report/* + - src/NET6CustomRuntime/Report/* - src/NET6MinimalAPI/Report/* - src/NET6MinimalAPIWebAdapter/Report/* + - src/NET6TopLevelStatements/Report/* + - src/NET6WithPowerTools/Report/* + - src/NET8/Report/* + - src/NET8MinimalAPI/Report/* + - src/NET8Native/Report/* + - src/NET8NativeMinimalAPI/Report/* + - loadtestcli/Report/* name: loadtest-$(date +%Y-%m-%H-%M) \ No newline at end of file diff --git a/loadtest/codebuild/load-test.yml b/loadtest/codebuild/load-test.yml new file mode 100755 index 0000000..75c7beb --- /dev/null +++ b/loadtest/codebuild/load-test.yml @@ -0,0 +1,32 @@ +config: + target: "{{ $processEnvironment.API_URL }}" + http: + timeout : 60 + processor: "../generator.js" + phases: + - duration: 60 + arrivalRate: 100 + +scenarios: + - name: "Generate products" + weight: 8 + flow: + - function: "generateProduct" + - put: + url: "/{{ Id }}" + headers: + Content-Type: "application/json" + json: + Id: "{{ Id }}" + Name: "{{ Name }}" + Price: "{{ Price }}" + - get: + url: "/{{ Id }}" + - think: 3 + - delete: + url: "/{{ Id }}" + - name: "Get products" + weight: 2 + flow: + - get: + url: "{{ $processEnvironment.API_URL }}" \ No newline at end of file diff --git a/loadtest/codebuild/run-all-load-tests.sh b/loadtest/codebuild/run-all-load-tests.sh index b2ac288..97e8813 100755 --- a/loadtest/codebuild/run-all-load-tests.sh +++ b/loadtest/codebuild/run-all-load-tests.sh @@ -2,6 +2,7 @@ TEST_DURATIOMN_SEC=60 LOG_INTERVAL_MIN=20 LOG_DELETE=yes DELETE_STACK=yes +ECR_URI=NotSet if [ "x${LT_TEST_DURATIOMN_SEC}" != x ]; then @@ -23,29 +24,135 @@ then DELETE_STACK=$LT_DELETE_STACK fi +if [ "x${LT_ECR_URI}" != x ]; +then + ECR_URI=$LT_ECR_URI +fi + + echo -------------------------------------------- echo TEST_DURATIOMN_SEC: $TEST_DURATIOMN_SEC echo LOG_INTERVAL_MIN: $LOG_INTERVAL_MIN echo LOG_DELETE: $LOG_DELETE echo DELETE_STACK: $DELETE_STACK +echo ECR_URI: $ECR_URI +echo LT_SNS_TOPIC_ARN: $LT_SNS_TOPIC_ARN echo -------------------------------------------- +if [ "$LT_NET6" != yes ]; +then + echo SKIPPING net6 - LT_NET6=$LT_NET6 +else + echo "RUNNING load test for net6" + cd ../../src/NET6/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET6_CONTAINERS" != yes ]; +then + echo SKIPPING net6 containers - LT_NET6_CONTAINERS=$LT_NET6_CONTAINERS +else + echo "RUNNING load test for net6 containers" + cd ../../src/NET6Containers/ + source ./deploy.sh $ECR_URI $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET6_CUSTOM" != yes ]; +then + echo SKIPPING net6 custom runtime - LT_NET6_CUSTOM=$LT_NET6_CUSTOM +else + echo "RUNNING load test for net6 custom runtime" + cd ../../src/NET6CustomRuntime/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + if [ "$LT_NET6_MINIMAL_API" != yes ]; then - echo SKIPPING net6 minimal api :$LT_NET6_MINIMAL_API + echo SKIPPING net6 minimal api - LT_NET6_MINIMAL_API=$LT_NET6_MINIMAL_API else echo "RUNNING load test for net6 minimal api" cd ../../src/NET6MinimalAPI/ source ./deploy.sh $DELETE_STACK - source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN fi if [ "$LT_NET6_MINIMAL_API_WEB_ADAPTER" != yes ]; then - echo SKIPPING net6 minimal api web adapter :$LT_NET6_MINIMAL_API_WEB_ADAPTER + echo SKIPPING net6 minimal api web adapter - LT_NET6_MINIMAL_API_WEB_ADAPTER = $LT_NET6_MINIMAL_API_WEB_ADAPTER else echo "RUNNING load test for net6 minimal api web adapter" cd ../../src/NET6MinimalAPIWebAdapter/ source ./deploy.sh $DELETE_STACK - source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN fi + +if [ "$LT_NET6_TOPLEVEL" != yes ]; +then + echo SKIPPING net6 top level - LT_NET6_TOPLEVEL = $LT_NET6_TOPLEVEL +else + echo "RUNNING load test for net6 top level" + cd ../../src/NET6TopLevelStatements/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET6_POWERTOOLS" != yes ]; +then + echo SKIPPING net6 power tools - LT_NET6_POWERTOOLS = $LT_NET6_POWERTOOLS +else + echo "RUNNING load test for net6 power tools" + cd ../../src/NET6WithPowerTools/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET8" != yes ]; +then + echo SKIPPING net8 - LT_NET8=$LT_NET8 +else + echo "RUNNING load test for net8" + cd ../../src/NET8/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET8_MINIMAL_API" != yes ]; +then + echo SKIPPING net8 minimal api - LT_NET8_MINIMAL_API=$LT_NET8_MINIMAL_API +else + echo "RUNNING load test for net8 minimal api" + cd ../../src/NET8MinimalAPI/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET8_NATIVE" != yes ]; +then + echo SKIPPING net8 native - LT_NET8_NATIVE=$LT_NET8_NATIVE +else + echo "RUNNING load test for net8 native" + cd ../../src/NET8Native/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +if [ "$LT_NET8_NATIVE_MINIMAL_API" != yes ]; +then + echo SKIPPING net8 native minimal api LT_NET8_NATIVE_MINIMAL_API=$LT_NET8_NATIVE_MINIMAL_API +else + echo "RUNNING load test for net8 native minimal api" + cd ../../src/NET8NativeMinimalAPI/ + source ./deploy.sh $DELETE_STACK + source ./run-loadtest.sh $TEST_DURATIOMN_SEC $LOG_INTERVAL_MIN $LOG_DELETE $LT_SNS_TOPIC_ARN +fi + +echo -------------------------------------------- +echo GENERATING FULL REPORT +echo -------------------------------------------- +cd ../../loadtestcli +dotnet run ../src $LT_SNS_TOPIC_ARN + + diff --git a/loadtest/codebuild/setup.sh b/loadtest/codebuild/setup.sh new file mode 100755 index 0000000..ffa1a0b --- /dev/null +++ b/loadtest/codebuild/setup.sh @@ -0,0 +1,16 @@ +#LT_NET_SDK_CHANNEL +#Specifies the source channel for the installation. The possible values are: +# STS: The most recent Standard Term Support release. +# LTS: The most recent Long Term Support release. +# Two-part version in A.B format, representing a specific release (for example, 3.1 or 6.0). +# Three-part version in A.B.Cxx format, representing a specific SDK release (for example, 6.0.1xx or 6.0.2xx). Available since the 5.0 release. +echo -------------------------------------------- +echo Installing dotnet runtime version: $LT_NET_SDK_CHANNEL +echo -------------------------------------------- + +curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel $LT_NET_SDK_CHANNEL + +echo -------------------------------------------- +echo Installing artillery +echo -------------------------------------------- +npm i artillery -g \ No newline at end of file diff --git a/loadtestcli/LoadtestCli.csproj b/loadtestcli/LoadtestCli.csproj new file mode 100644 index 0000000..0f937ed --- /dev/null +++ b/loadtestcli/LoadtestCli.csproj @@ -0,0 +1,15 @@ + + + + + + + + + Exe + net6.0 + enable + enable + + + diff --git a/loadtestcli/Program.cs b/loadtestcli/Program.cs new file mode 100644 index 0000000..0a71e71 --- /dev/null +++ b/loadtestcli/Program.cs @@ -0,0 +1,390 @@ +using System; +using System.Text; +using System.Text.Json; +using System.Threading.Tasks; +using Amazon.SimpleNotificationService; +using Amazon.SimpleNotificationService.Model; + +public record QueryResultWrapper +{ + public string? LoadTestType { get; set; } + + public QueryResult? ColdStart { get; set; } + + public QueryResult? WarmStart { get; set; } + + public string? ErrorCount { get; set; } +} + +public record QueryResult +{ + public string? Count { get; set; } + + public string? P50 { get; set; } + + public string? P90 { get; set; } + + public string? P99 { get; set; } + + public string? Max { get; set; } +} + +public class CloudWatchMetricResultWrapper +{ + public string? Label { get; set; } + public List? Datapoints { get; set; } +} +public class CloudWatchMetricDatapoint +{ + public DateTime? Timestamp { get; set; } + public double? Sum { get; set; } + public string? Unit { get; set; } +} + + + +public class LoadTestResults +{ + public List>? results { get; set; } +} + +public class LoadTestSubResult +{ + public string? field { get; set; } + public string? value { get; set; } +} + +internal class Program +{ + private static async Task Main(string[] args) + { + var folderToAnalize = ""; + var reportFileOutputPath = ""; + var snsTopicArn=""; + + var reportResultList = new List(); + + DisplayStartMsg(); + + if(args.Count()<1) + { + DisplayHelp(); + return -1; + } + folderToAnalize=args[0]; + + if(args.Count()>=2) + { + snsTopicArn=args[1]; + } + + if(args.Count()>=3) + { + reportFileOutputPath=args[2]; + } + else + { + reportFileOutputPath=$"./Report/{DateTime.UtcNow:yyyy-MM-dd-hh-mm}-loadtest-report.html"; + } + + try + { + System.Console.WriteLine($"STARTING using folder: {folderToAnalize}"); + System.Console.WriteLine($" - using folder: {folderToAnalize}"); + System.Console.WriteLine($" - destination path: {reportFileOutputPath}"); + + if(!Directory.Exists(folderToAnalize)) + { + System.Console.WriteLine($"ERROR: folder {folderToAnalize} does not exist"); + return -2; + } + foreach (var folderPath in Directory.EnumerateDirectories(folderToAnalize)) + { + System.Console.WriteLine($"Processing folder: {folderPath}"); + var reportFolderPath = Path.Combine(folderPath, "Report"); + System.Console.WriteLine($" - Searching sub folder {reportFolderPath}"); + if (Directory.Exists(reportFolderPath)) + { + System.Console.WriteLine($" - Processing sub folder: {reportFolderPath}"); + var reportFolderInfo = new DirectoryInfo(reportFolderPath); + foreach (var reportFileInfo in reportFolderInfo.GetFiles("load-test-report-*.json")) + { + QueryResultWrapper ltResult; + try + { + System.Console.WriteLine($" - Processing sub folder file: {reportFileInfo.Name}"); + var ltName = reportFileInfo.Name.Replace("load-test-report-", "").Replace(reportFileInfo.Extension,""); + var title = $"{reportFolderInfo.Parent?.Name} - {ltName}"; + System.Console.WriteLine($" - Report: {title}"); + ltResult = await DeserializeLoadTestResultAsync(reportFileInfo.FullName, title); + ltResult.ErrorCount=await GetLoadtestErrorCountAsync(reportFileInfo.FullName.Replace("load-test-report-","load-test-errors-")); + } + catch(Exception ex) + { + ltResult=new QueryResultWrapper() + { + LoadTestType= $"error processing file {reportFileInfo.FullName}: {ex.Message}" + }; + System.Console.WriteLine($" !!!! ERROR PROCESSING FILE: {reportFileInfo.FullName}: {ex.Message} !!!!"); + System.Console.WriteLine(ex.ToString()); + } + reportResultList.Add(ltResult); + } + } + } + System.Console.WriteLine("Generating HTML Report!"); + GenerateHtmlGlobalReport(reportResultList,reportFileOutputPath,snsTopicArn); + System.Console.WriteLine("FINISHED!"); + } + catch(Exception ex) + { + System.Console.WriteLine($"ERROR!:{ex.Message}"); + System.Console.WriteLine($"ERROR!:{ex.ToString()}"); + } + return 0; + } + + private static void DisplayStartMsg() + { + System.Console.WriteLine("LOAD-TEST CLI"); + System.Console.WriteLine("--------------------------------------"); + } + + private static void DisplayHelp() + { + System.Console.WriteLine("ERROR - Missing required parameters:"); + System.Console.WriteLine("- param1: mandatory folder to process"); + System.Console.WriteLine("- param2: optional sns arn"); + System.Console.WriteLine("- param3: optional output file path"); + } + private static void GenerateHtmlGlobalReport(List reportResultList,string reportFileOutputPath,string snsTopicArn) + { + /* + we are producing an html similar to this: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Cold Start (ms)Warm Start (ms)
Invoke Countp50p90p99maxInvoke Countp50p90p99max
Minimal API on ARM642421972.792049.162107.322124.55136,8166.019.3724.69331.6
+ */ + var sb = new StringBuilder(); + + sb.Append(@" + + + + + + + + + + + + + + + + + + + + "); + foreach (var ltResult in reportResultList) + { + sb.Append($@" + + + + + + + + + + + + + + "); + } + sb.Append($@" +
Cold Start (ms)Warm Start (ms)
Invoke Countp50p90p99maxInvoke Countp50p90p99max
{ltResult.LoadTestType} errors:{ltResult.ErrorCount}{FormatCount(ltResult.ColdStart?.Count??"-")}{FormatMilli(ltResult.ColdStart?.P50??"-")}{FormatMilli(ltResult.ColdStart?.P90??"-")}{FormatMilli(ltResult.ColdStart?.P99??"-")}{FormatMilli(ltResult.ColdStart?.Max??"-")}{FormatCount(ltResult.WarmStart?.Count??"-")}{FormatMilli(ltResult.WarmStart?.P50??"-")}{FormatMilli(ltResult.WarmStart?.P90??"-")}{FormatMilli(ltResult.WarmStart?.P99??"-")}{FormatMilli(ltResult.WarmStart?.Max??"-")}
+ "); + + var outputDir=Path.GetDirectoryName(reportFileOutputPath); + if(!String.IsNullOrEmpty(outputDir)) + { + System.Console.WriteLine($"CREATING OUTPUT FOLDER:{outputDir}"); + Directory.CreateDirectory(outputDir); + } + File.WriteAllText(reportFileOutputPath, sb.ToString()); + + if(!string.IsNullOrEmpty(snsTopicArn)) + { + SendSNSMsg(snsTopicArn,"servereless dotnet demo - LOAD TEST FINAL REPORT",sb.ToString()); + } + else + { + Console.WriteLine("SKIPPING SENSID SNS MSG"); + } + System.Console.WriteLine($"OUTPUT REPORT CREATED: {reportFileOutputPath}"); + } + + private static async Task GetLoadtestErrorCountAsync(string filePath) + { + int ret=-1; + System.Console.WriteLine($" - PROCESSING ERROR ERROR COUNT FILE:{filePath}"); + try + { + if(!File.Exists(filePath)) + return "missing"; + + var json = await File.ReadAllTextAsync(filePath); + var tmp=JsonSerializer.Deserialize(json); + if(tmp!=null) + { + if(tmp.Datapoints!=null) + { + ret=0; + foreach(var i in tmp.Datapoints) + { + ret+=(int)(i.Sum??0); + } + } + } + else + return "invalid"; + } + catch(Exception ex) + { + System.Console.WriteLine($"FAILED TO PROCESS ERROR COUNT FILE!:{ex.Message}"); + System.Console.WriteLine($"{ex.ToString()}"); + return $"failed: {ex.Message}"; + } + return ret.ToString(); + } + + private static async Task DeserializeLoadTestResultAsync(string filePath,string title) + { + var json = await File.ReadAllTextAsync(filePath); + + var ret = new QueryResultWrapper + { + LoadTestType = title, + WarmStart=null, + ColdStart=null + }; + + var lt=JsonSerializer.Deserialize(json); + + if(lt?.results==null) + return ret; + + foreach (var ltItem in lt.results) + { + var isWarm = ltItem.Where(r => r.field == "coldstart" && r.value == "0").Count() > 0; + System.Console.WriteLine($" . isWarm: {isWarm}"); + + var tmp = new QueryResult + { + Count = ltItem?.Where(r => r.field == "count").FirstOrDefault()?.value, + P50 = ltItem?.Where(r => r.field == "p50").FirstOrDefault()?.value, + P90 = ltItem?.Where(r => r.field == "p90").FirstOrDefault()?.value, + P99 = ltItem?.Where(r => r.field == "p99").FirstOrDefault()?.value, + Max = ltItem?.Where(r => r.field == "max").FirstOrDefault()?.value + }; + + if (isWarm) + ret.WarmStart = tmp; + else + ret.ColdStart = tmp; + } + + if(ret.WarmStart==null) + { + System.Console.WriteLine(" !!!! ATTENTION MISING WARM STATS !!!!"); + } + + if(ret.ColdStart==null) + { + System.Console.WriteLine(" !!!! ATTENTION MISING COLD STATS !!!!"); + } + + return ret; + } + + private static string FormatMilli(string value) + { + Double valueNr=0; + if(!Double.TryParse(value,out valueNr)) + return value; + return ($"{valueNr:#,##0.00}"); + } + + private static string FormatCount(string value) + { + Double valueNr=0; + if(!Double.TryParse(value,out valueNr)) + return value; + return ($"{valueNr:#,##0}"); + } + + private static void SendSNSMsg(string snsTopicArn,string subject,string msg) + { + Console.WriteLine($"SENDING SNS MSG: {snsTopicArn}"); + try + { + using(var snsClient = new AmazonSimpleNotificationServiceClient()) + { + var request = new PublishRequest + { + TopicArn = snsTopicArn, + Subject=subject, + Message = msg, + }; + + var t = snsClient.PublishAsync(request); + var result=t.Result; + + Console.WriteLine($"SNS Message Published ID:{result.MessageId}"); + + } + } + catch(Exception ex) + { + System.Console.WriteLine($"FAILED TO SEND SNS MSG!:{ex.Message}"); + System.Console.WriteLine($"{ex.ToString()}"); + } + } +} \ No newline at end of file diff --git a/loadtestcli/Samples/EMPTY/Report/tmp.txt b/loadtestcli/Samples/EMPTY/Report/tmp.txt new file mode 100644 index 0000000..945c9b4 --- /dev/null +++ b/loadtestcli/Samples/EMPTY/Report/tmp.txt @@ -0,0 +1 @@ +. \ No newline at end of file diff --git a/loadtestcli/Samples/FAKE/Report/load-test-report-Empty.json b/loadtestcli/Samples/FAKE/Report/load-test-report-Empty.json new file mode 100644 index 0000000..e69de29 diff --git a/loadtestcli/Samples/FAKE/Report/load-test-report-No-Cold.json b/loadtestcli/Samples/FAKE/Report/load-test-report-No-Cold.json new file mode 100644 index 0000000..0d462b1 --- /dev/null +++ b/loadtestcli/Samples/FAKE/Report/load-test-report-No-Cold.json @@ -0,0 +1,36 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "139116" + }, + { + "field": "p50", + "value": "6.1078" + }, + { + "field": "p90", + "value": "9.6783" + }, + { + "field": "p99", + "value": "23.9189" + }, + { + "field": "max", + "value": "135.26" + } + ] + ], + "statistics": { + "recordsMatched": 140114.0, + "recordsScanned": 529742.0, + "bytesScanned": 176419355.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/FAKE/Report/load-test-report-No-Warm.json b/loadtestcli/Samples/FAKE/Report/load-test-report-No-Warm.json new file mode 100644 index 0000000..1c47e7d --- /dev/null +++ b/loadtestcli/Samples/FAKE/Report/load-test-report-No-Warm.json @@ -0,0 +1,36 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "802" + }, + { + "field": "p50", + "value": "1477.8616" + }, + { + "field": "p90", + "value": "1533.5383" + }, + { + "field": "p99", + "value": "1666.1844" + }, + { + "field": "max", + "value": "1836.55" + } + ] + ], + "statistics": { + "recordsMatched": 137744.0, + "recordsScanned": 520790.0, + "bytesScanned": 173060538.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/FAKE/Report/load-test-report-Null.json b/loadtestcli/Samples/FAKE/Report/load-test-report-Null.json new file mode 100644 index 0000000..2c63c08 --- /dev/null +++ b/loadtestcli/Samples/FAKE/Report/load-test-report-Null.json @@ -0,0 +1,2 @@ +{ +} diff --git a/loadtestcli/Samples/NET6/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..4ea0b92 --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-errors-arm64.json @@ -0,0 +1,10 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:29:00+00:00", + "Sum": 0.0, + "Unit": "Count" + } + ] +} diff --git a/loadtestcli/Samples/NET6/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6/Report/load-test-errors-x86.json new file mode 100644 index 0000000..15a60ff --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-errors-x86.json @@ -0,0 +1,10 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:27:00+00:00", + "Sum": 0.0, + "Unit": "Count" + } + ] +} diff --git a/loadtestcli/Samples/NET6/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6/Report/load-test-report-arm64.json new file mode 100644 index 0000000..ac4021b --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "137147" + }, + { + "field": "p50", + "value": "6.0116" + }, + { + "field": "p90", + "value": "8.7991" + }, + { + "field": "p99", + "value": "19.4591" + }, + { + "field": "max", + "value": "218.59" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "567" + }, + { + "field": "p50", + "value": "1121.5751" + }, + { + "field": "p90", + "value": "1174.3456" + }, + { + "field": "p99", + "value": "1236.9951" + }, + { + "field": "max", + "value": "1303.02" + } + ] + ], + "statistics": { + "recordsMatched": 137714.0, + "recordsScanned": 434509.0, + "bytesScanned": 59860688.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..0aa2af5 --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 18:58:34 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 137147 +RESULTS p50 6.0116 +RESULTS p90 8.7991 +RESULTS p99 19.4591 +RESULTS max 218.59 +RESULTS coldstart 1 +RESULTS count 567 +RESULTS p50 1121.5751 +RESULTS p90 1174.3456 +RESULTS p99 1236.9951 +RESULTS max 1303.02 +STATISTICS 59860688.0 137714.0 434509.0 diff --git a/loadtestcli/Samples/NET6/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6/Report/load-test-report-x86.json new file mode 100644 index 0000000..649da94 --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "135097" + }, + { + "field": "p50", + "value": "5.3794" + }, + { + "field": "p90", + "value": "9.3759" + }, + { + "field": "p99", + "value": "21.746" + }, + { + "field": "max", + "value": "134.11" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "468" + }, + { + "field": "p50", + "value": "865.7695" + }, + { + "field": "p90", + "value": "928.5124" + }, + { + "field": "p99", + "value": "1041.6136" + }, + { + "field": "max", + "value": "1125.25" + } + ] + ], + "statistics": { + "recordsMatched": 135565.0, + "recordsScanned": 428084.0, + "bytesScanned": 59006872.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6/Report/load-test-report-x86.txt new file mode 100644 index 0000000..3124fce --- /dev/null +++ b/loadtestcli/Samples/NET6/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 18:47:49 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 135097 +RESULTS p50 5.3794 +RESULTS p90 9.3759 +RESULTS p99 21.746 +RESULTS max 134.11 +RESULTS coldstart 1 +RESULTS count 468 +RESULTS p50 865.7695 +RESULTS p90 928.5124 +RESULTS p99 1041.6136 +RESULTS max 1125.25 +STATISTICS 59006872.0 135565.0 428084.0 diff --git a/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..4cecab3 --- /dev/null +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-arm64.json @@ -0,0 +1,10 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:29:00+00:00", + "Sum": 100.0, + "Unit": "Count" + } + ] +} diff --git a/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-x86.json new file mode 100644 index 0000000..c746834 --- /dev/null +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-errors-x86.json @@ -0,0 +1,15 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:27:00+00:00", + "Sum": 0.0, + "Unit": "Count" + }, + { + "Timestamp": "2023-11-15T12:27:00+00:00", + "Sum": 20.0, + "Unit": "Count" + } + ] +} diff --git a/src/NET6MinimalAPI/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.json similarity index 74% rename from src/NET6MinimalAPI/Report/load-test-report-arm64.json rename to loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.json index 005bf4d..03b0870 100644 --- a/src/NET6MinimalAPI/Report/load-test-report-arm64.json +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.json @@ -7,7 +7,7 @@ }, { "field": "count", - "value": "135963" + "value": "135961" }, { "field": "p50", @@ -19,11 +19,11 @@ }, { "field": "p99", - "value": "20.0868" + "value": "21.0665" }, { "field": "max", - "value": "528.13" + "value": "182.08" } ], [ @@ -33,30 +33,30 @@ }, { "field": "count", - "value": "246" + "value": "1151" }, { "field": "p50", - "value": "2105.2185" + "value": "1951.2247" }, { "field": "p90", - "value": "2164.9673" + "value": "2028.7864" }, { "field": "p99", - "value": "2215.3132" + "value": "2105.2185" }, { "field": "max", - "value": "2228.18" + "value": "2189.86" } ] ], "statistics": { - "recordsMatched": 136209.0, - "recordsScanned": 1226110.0, - "bytesScanned": 900312482.0 + "recordsMatched": 137112.0, + "recordsScanned": 519167.0, + "bytesScanned": 173196929.0 }, "status": "Complete" } diff --git a/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..c5b1950 --- /dev/null +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 19:25:15 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 135961 +RESULTS p50 6.2055 +RESULTS p90 9.6783 +RESULTS p99 21.0665 +RESULTS max 182.08 +RESULTS coldstart 1 +RESULTS count 1151 +RESULTS p50 1951.2247 +RESULTS p90 2028.7864 +RESULTS p99 2105.2185 +RESULTS max 2189.86 +STATISTICS 173196929.0 137112.0 519167.0 diff --git a/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.json new file mode 100644 index 0000000..51b88b3 --- /dev/null +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "134829" + }, + { + "field": "p50", + "value": "5.9169" + }, + { + "field": "p90", + "value": "10.6454" + }, + { + "field": "p99", + "value": "23.9189" + }, + { + "field": "max", + "value": "186.97" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "858" + }, + { + "field": "p50", + "value": "1553.5943" + }, + { + "field": "p90", + "value": "1666.1844" + }, + { + "field": "p99", + "value": "1837.6493" + }, + { + "field": "max", + "value": "2335.72" + } + ] + ], + "statistics": { + "recordsMatched": 135687.0, + "recordsScanned": 512095.0, + "bytesScanned": 169019404.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.txt new file mode 100644 index 0000000..ace00da --- /dev/null +++ b/loadtestcli/Samples/NET6CustomRuntime/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 19:14:30 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 134829 +RESULTS p50 5.9169 +RESULTS p90 10.6454 +RESULTS p99 23.9189 +RESULTS max 186.97 +RESULTS coldstart 1 +RESULTS count 858 +RESULTS p50 1553.5943 +RESULTS p90 1666.1844 +RESULTS p99 1837.6493 +RESULTS max 2335.72 +STATISTICS 169019404.0 135687.0 512095.0 diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..2c63c08 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-arm64.json @@ -0,0 +1,2 @@ +{ +} diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-x86.json new file mode 100644 index 0000000..2c63c08 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-errors-x86.json @@ -0,0 +1,2 @@ +{ +} diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.json new file mode 100644 index 0000000..5024ee6 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "139157" + }, + { + "field": "p50", + "value": "6.2055" + }, + { + "field": "p90", + "value": "9.5259" + }, + { + "field": "p99", + "value": "20.7347" + }, + { + "field": "max", + "value": "186.95" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "243" + }, + { + "field": "p50", + "value": "2107.3237" + }, + { + "field": "p90", + "value": "2195.4747" + }, + { + "field": "p99", + "value": "2289.6004" + }, + { + "field": "max", + "value": "2309.17" + } + ] + ], + "statistics": { + "recordsMatched": 139400.0, + "recordsScanned": 1254913.0, + "bytesScanned": 921494250.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..15d2922 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 19:48:11 UTC 2023 +arm64 RESULTS lambda: Net6-MinimalApi-Arm64 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 139157 +RESULTS p50 6.2055 +RESULTS p90 9.5259 +RESULTS p99 20.7347 +RESULTS max 186.95 +RESULTS coldstart 1 +RESULTS count 243 +RESULTS p50 2107.3237 +RESULTS p90 2195.4747 +RESULTS p99 2289.6004 +RESULTS max 2309.17 +STATISTICS 921494250.0 139400.0 1254913.0 diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.json new file mode 100644 index 0000000..abb6b80 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "139442" + }, + { + "field": "p50", + "value": "6.3048" + }, + { + "field": "p90", + "value": "10.8157" + }, + { + "field": "p99", + "value": "24.6905" + }, + { + "field": "max", + "value": "168.18" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "201" + }, + { + "field": "p50", + "value": "1756.8276" + }, + { + "field": "p90", + "value": "1884.1461" + }, + { + "field": "p99", + "value": "2084.2817" + }, + { + "field": "max", + "value": "2446.27" + } + ] + ], + "statistics": { + "recordsMatched": 139643.0, + "recordsScanned": 1257061.0, + "bytesScanned": 918027433.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.txt new file mode 100644 index 0000000..077753c --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPI/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 19:37:29 UTC 2023 +x86 RESULTS lambda: Net6-MinimalApi-X86 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 139442 +RESULTS p50 6.3048 +RESULTS p90 10.8157 +RESULTS p99 24.6905 +RESULTS max 168.18 +RESULTS coldstart 1 +RESULTS count 201 +RESULTS p50 1756.8276 +RESULTS p90 1884.1461 +RESULTS p99 2084.2817 +RESULTS max 2446.27 +STATISTICS 918027433.0 139643.0 1257061.0 diff --git a/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..e69de29 diff --git a/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-errors-x86.json new file mode 100644 index 0000000..e69de29 diff --git a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json similarity index 71% rename from src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json rename to loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json index be65931..f741b2e 100644 --- a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json +++ b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.json @@ -7,7 +7,7 @@ }, { "field": "count", - "value": "138527" + "value": "138066" }, { "field": "p50", @@ -19,11 +19,11 @@ }, { "field": "p99", - "value": "17.9744" + "value": "20.0868" }, { "field": "max", - "value": "838.78" + "value": "188.1" } ], [ @@ -33,30 +33,30 @@ }, { "field": "count", - "value": "139" + "value": "149" }, { "field": "p50", - "value": "1277.1986" + "value": "1288.7395" }, { "field": "p90", - "value": "1326.6409" + "value": "1335.9553" }, { "field": "p99", - "value": "1358.8491" + "value": "1428.4831" }, { "field": "max", - "value": "1367.49" + "value": "1464.5" } ] ], "statistics": { - "recordsMatched": 138666.0, - "recordsScanned": 694392.0, - "bytesScanned": 268615241.0 + "recordsMatched": 138215.0, + "recordsScanned": 692169.0, + "bytesScanned": 267841605.0 }, "status": "Complete" } diff --git a/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..8860889 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 20:10:50 UTC 2023 +arm64 RESULTS lambda: Net6-MinimalApi-WebadApter-Arm64 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 138066 +RESULTS p50 6.1078 +RESULTS p90 9.3759 +RESULTS p99 20.0868 +RESULTS max 188.1 +RESULTS coldstart 1 +RESULTS count 149 +RESULTS p50 1288.7395 +RESULTS p90 1335.9553 +RESULTS p99 1428.4831 +RESULTS max 1464.5 +STATISTICS 267841605.0 138215.0 692169.0 diff --git a/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json new file mode 100644 index 0000000..06b9194 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "134610" + }, + { + "field": "p50", + "value": "6.1078" + }, + { + "field": "p90", + "value": "9.9905" + }, + { + "field": "p99", + "value": "21.0665" + }, + { + "field": "max", + "value": "123.79" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "112" + }, + { + "field": "p50", + "value": "1028.1669" + }, + { + "field": "p90", + "value": "1109.3115" + }, + { + "field": "p99", + "value": "1210.0916" + }, + { + "field": "max", + "value": "1237.92" + } + ] + ], + "statistics": { + "recordsMatched": 134722.0, + "recordsScanned": 674331.0, + "bytesScanned": 260957154.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt new file mode 100644 index 0000000..0f24e84 --- /dev/null +++ b/loadtestcli/Samples/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 20:00:08 UTC 2023 +x86 RESULTS lambda: Net6-MinimalApi-WebadApter-X86 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 134610 +RESULTS p50 6.1078 +RESULTS p90 9.9905 +RESULTS p99 21.0665 +RESULTS max 123.79 +RESULTS coldstart 1 +RESULTS count 112 +RESULTS p50 1028.1669 +RESULTS p90 1109.3115 +RESULTS p99 1210.0916 +RESULTS max 1237.92 +STATISTICS 260957154.0 134722.0 674331.0 diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6Native/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..e466dcb --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-errors-arm64.json @@ -0,0 +1 @@ +invalid \ No newline at end of file diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6Native/Report/load-test-errors-x86.json new file mode 100644 index 0000000..e466dcb --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-errors-x86.json @@ -0,0 +1 @@ +invalid \ No newline at end of file diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.json new file mode 100644 index 0000000..600f742 --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.json @@ -0,0 +1,36 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "137737" + }, + { + "field": "p50", + "value": "3.4491" + }, + { + "field": "p90", + "value": "5.8237" + }, + { + "field": "p99", + "value": "14.3927" + }, + { + "field": "max", + "value": "64.44" + } + ] + ], + "statistics": { + "recordsMatched": 137737.0, + "recordsScanned": 690625.0, + "bytesScanned": 97122238.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..9b689a2 --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-report-arm64.txt @@ -0,0 +1,12 @@ +Mon Nov 13 20:37:16 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 137737 +RESULTS p50 3.4491 +RESULTS p90 5.8237 +RESULTS p99 14.3927 +RESULTS max 64.44 +STATISTICS 97122238.0 137737.0 690625.0 diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.json new file mode 100644 index 0000000..b0a54b9 --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "136565" + }, + { + "field": "p50", + "value": "5.9169" + }, + { + "field": "p90", + "value": "9.8332" + }, + { + "field": "p99", + "value": "22.0939" + }, + { + "field": "max", + "value": "443.61" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "886" + }, + { + "field": "p50", + "value": "1610.5137" + }, + { + "field": "p90", + "value": "1739.3556" + }, + { + "field": "p99", + "value": "1918.3506" + }, + { + "field": "max", + "value": "2234.13" + } + ] + ], + "statistics": { + "recordsMatched": 137451.0, + "recordsScanned": 434767.0, + "bytesScanned": 59976637.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.txt new file mode 100644 index 0000000..63d1de9 --- /dev/null +++ b/loadtestcli/Samples/NET6Native/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 20:26:31 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 136565 +RESULTS p50 5.9169 +RESULTS p90 9.8332 +RESULTS p99 22.0939 +RESULTS max 443.61 +RESULTS coldstart 1 +RESULTS count 886 +RESULTS p50 1610.5137 +RESULTS p90 1739.3556 +RESULTS p99 1918.3506 +RESULTS max 2234.13 +STATISTICS 59976637.0 137451.0 434767.0 diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-arm64.json b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-arm64.json new file mode 100644 index 0000000..4ea0b92 --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-arm64.json @@ -0,0 +1,10 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:29:00+00:00", + "Sum": 0.0, + "Unit": "Count" + } + ] +} diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-x86.json b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-x86.json new file mode 100644 index 0000000..54147f5 --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-errors-x86.json @@ -0,0 +1,15 @@ +{ + "Label": "Errors", + "Datapoints": [ + { + "Timestamp": "2023-11-15T12:27:00+00:00", + "Sum": 0.0, + "Unit": "Count" + }, + { + "Timestamp": "2023-11-15T12:27:00+00:00", + "Sum": 0.0, + "Unit": "Count" + } + ] +} diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.json new file mode 100644 index 0000000..cd10bfa --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "138920" + }, + { + "field": "p50", + "value": "5.9115" + }, + { + "field": "p90", + "value": "9.4595" + }, + { + "field": "p99", + "value": "20.006" + }, + { + "field": "max", + "value": "148.55" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "614" + }, + { + "field": "p50", + "value": "1154.5603" + }, + { + "field": "p90", + "value": "1199.2551" + }, + { + "field": "p99", + "value": "1256.9361" + }, + { + "field": "max", + "value": "1452.91" + } + ] + ], + "statistics": { + "recordsMatched": 139534.0, + "recordsScanned": 440180.0, + "bytesScanned": 60681734.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..1b5eb8f --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 21:01:12 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 138920 +RESULTS p50 5.9115 +RESULTS p90 9.4595 +RESULTS p99 20.006 +RESULTS max 148.55 +RESULTS coldstart 1 +RESULTS count 614 +RESULTS p50 1154.5603 +RESULTS p90 1199.2551 +RESULTS p99 1256.9361 +RESULTS max 1452.91 +STATISTICS 60681734.0 139534.0 440180.0 diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.json new file mode 100644 index 0000000..6c61d15 --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "138755" + }, + { + "field": "p50", + "value": "5.5529" + }, + { + "field": "p90", + "value": "10.1504" + }, + { + "field": "p99", + "value": "22.4474" + }, + { + "field": "max", + "value": "171.14" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "485" + }, + { + "field": "p50", + "value": "924.8077" + }, + { + "field": "p90", + "value": "997.795" + }, + { + "field": "p99", + "value": "1118.2171" + }, + { + "field": "max", + "value": "1252.6" + } + ] + ], + "statistics": { + "recordsMatched": 139240.0, + "recordsScanned": 440977.0, + "bytesScanned": 60856076.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.txt new file mode 100644 index 0000000..817111b --- /dev/null +++ b/loadtestcli/Samples/NET6TopLevelStatements/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 20:50:27 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 138755 +RESULTS p50 5.5529 +RESULTS p90 10.1504 +RESULTS p99 22.4474 +RESULTS max 171.14 +RESULTS coldstart 1 +RESULTS count 485 +RESULTS p50 924.8077 +RESULTS p90 997.795 +RESULTS p99 1118.2171 +RESULTS max 1252.6 +STATISTICS 60856076.0 139240.0 440977.0 diff --git a/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.json new file mode 100644 index 0000000..b068b00 --- /dev/null +++ b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "132463" + }, + { + "field": "p50", + "value": "6.2506" + }, + { + "field": "p90", + "value": "9.4595" + }, + { + "field": "p99", + "value": "19.8472" + }, + { + "field": "max", + "value": "205.08" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "618" + }, + { + "field": "p50", + "value": "1154.5603" + }, + { + "field": "p90", + "value": "1195.6645" + }, + { + "field": "p99", + "value": "1240.7098" + }, + { + "field": "max", + "value": "1309.71" + } + ] + ], + "statistics": { + "recordsMatched": 133081.0, + "recordsScanned": 543411.0, + "bytesScanned": 97004522.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..2248cdd --- /dev/null +++ b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 21:25:42 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 132463 +RESULTS p50 6.2506 +RESULTS p90 9.4595 +RESULTS p99 19.8472 +RESULTS max 205.08 +RESULTS coldstart 1 +RESULTS count 618 +RESULTS p50 1154.5603 +RESULTS p90 1195.6645 +RESULTS p99 1240.7098 +RESULTS max 1309.71 +STATISTICS 97004522.0 133081.0 543411.0 diff --git a/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.json b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.json new file mode 100644 index 0000000..1488688 --- /dev/null +++ b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "138929" + }, + { + "field": "p50", + "value": "6.0116" + }, + { + "field": "p90", + "value": "9.5259" + }, + { + "field": "p99", + "value": "20.7347" + }, + { + "field": "max", + "value": "170.67" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "493" + }, + { + "field": "p50", + "value": "885.9028" + }, + { + "field": "p90", + "value": "935.9666" + }, + { + "field": "p99", + "value": "1010.8444" + }, + { + "field": "max", + "value": "1110.18" + } + ] + ], + "statistics": { + "recordsMatched": 139422.0, + "recordsScanned": 568395.0, + "bytesScanned": 101301803.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.txt new file mode 100644 index 0000000..5c44d25 --- /dev/null +++ b/loadtestcli/Samples/NET6WithPowerTools/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 21:14:57 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 138929 +RESULTS p50 6.0116 +RESULTS p90 9.5259 +RESULTS p99 20.7347 +RESULTS max 170.67 +RESULTS coldstart 1 +RESULTS count 493 +RESULTS p50 885.9028 +RESULTS p90 935.9666 +RESULTS p99 1010.8444 +RESULTS max 1110.18 +STATISTICS 101301803.0 139422.0 568395.0 diff --git a/loadtestcli/Samples/NET8/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET8/Report/load-test-report-arm64.json new file mode 100644 index 0000000..68a0455 --- /dev/null +++ b/loadtestcli/Samples/NET8/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "139116" + }, + { + "field": "p50", + "value": "6.1078" + }, + { + "field": "p90", + "value": "9.6783" + }, + { + "field": "p99", + "value": "23.9189" + }, + { + "field": "max", + "value": "135.26" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "998" + }, + { + "field": "p50", + "value": "1716.9015" + }, + { + "field": "p90", + "value": "1779.8039" + }, + { + "field": "p99", + "value": "1839.487" + }, + { + "field": "max", + "value": "1856.62" + } + ] + ], + "statistics": { + "recordsMatched": 140114.0, + "recordsScanned": 529742.0, + "bytesScanned": 176419355.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET8/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET8/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..b95d153 --- /dev/null +++ b/loadtestcli/Samples/NET8/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 07:09:45 UTC 2023 +arm64 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 139116 +RESULTS p50 6.1078 +RESULTS p90 9.6783 +RESULTS p99 23.9189 +RESULTS max 135.26 +RESULTS coldstart 1 +RESULTS count 998 +RESULTS p50 1716.9015 +RESULTS p90 1779.8039 +RESULTS p99 1839.487 +RESULTS max 1856.62 +STATISTICS 176419355.0 140114.0 529742.0 diff --git a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json b/loadtestcli/Samples/NET8/Report/load-test-report-x86.json similarity index 71% rename from src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json rename to loadtestcli/Samples/NET8/Report/load-test-report-x86.json index 60522ca..c1e6979 100644 --- a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.json +++ b/loadtestcli/Samples/NET8/Report/load-test-report-x86.json @@ -7,7 +7,7 @@ }, { "field": "count", - "value": "140872" + "value": "136942" }, { "field": "p50", @@ -19,11 +19,11 @@ }, { "field": "p99", - "value": "21.746" + "value": "23.9189" }, { "field": "max", - "value": "154.62" + "value": "143.08" } ], [ @@ -33,30 +33,30 @@ }, { "field": "count", - "value": "112" + "value": "802" }, { "field": "p50", - "value": "1013.88" + "value": "1477.8616" }, { "field": "p90", - "value": "1102.6789" + "value": "1533.5383" }, { "field": "p99", - "value": "1330.6248" + "value": "1666.1844" }, { "field": "max", - "value": "1392.85" + "value": "1836.55" } ] ], "statistics": { - "recordsMatched": 140984.0, - "recordsScanned": 705687.0, - "bytesScanned": 272986184.0 + "recordsMatched": 137744.0, + "recordsScanned": 520790.0, + "bytesScanned": 173060538.0 }, "status": "Complete" } diff --git a/loadtestcli/Samples/NET8/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET8/Report/load-test-report-x86.txt new file mode 100644 index 0000000..b86b219 --- /dev/null +++ b/loadtestcli/Samples/NET8/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 06:58:59 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 136942 +RESULTS p50 6.2055 +RESULTS p90 10.3128 +RESULTS p99 23.9189 +RESULTS max 143.08 +RESULTS coldstart 1 +RESULTS count 802 +RESULTS p50 1477.8616 +RESULTS p90 1533.5383 +RESULTS p99 1666.1844 +RESULTS max 1836.55 +STATISTICS 173060538.0 137744.0 520790.0 diff --git a/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.json b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.json new file mode 100644 index 0000000..a38559c --- /dev/null +++ b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "136380" + }, + { + "field": "p50", + "value": "6.3048" + }, + { + "field": "p90", + "value": "10.6454" + }, + { + "field": "p99", + "value": "30.8348" + }, + { + "field": "max", + "value": "156.29" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "262" + }, + { + "field": "p50", + "value": "1990.6222" + }, + { + "field": "p90", + "value": "2088.4523" + }, + { + "field": "p99", + "value": "2156.3291" + }, + { + "field": "max", + "value": "2159.3" + } + ] + ], + "statistics": { + "recordsMatched": 136642.0, + "recordsScanned": 1230214.0, + "bytesScanned": 943182997.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.txt b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.txt new file mode 100644 index 0000000..3a4b8c8 --- /dev/null +++ b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-arm64.txt @@ -0,0 +1,18 @@ +Mon Nov 13 07:33:34 UTC 2023 +arm64 RESULTS lambda: Net8-MinimalApi-Arm64 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 136380 +RESULTS p50 6.3048 +RESULTS p90 10.6454 +RESULTS p99 30.8348 +RESULTS max 156.29 +RESULTS coldstart 1 +RESULTS count 262 +RESULTS p50 1990.6222 +RESULTS p90 2088.4523 +RESULTS p99 2156.3291 +RESULTS max 2159.3 +STATISTICS 943182997.0 136642.0 1230214.0 diff --git a/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.json b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.json new file mode 100644 index 0000000..b834f98 --- /dev/null +++ b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "141497" + }, + { + "field": "p50", + "value": "5.8237" + }, + { + "field": "p90", + "value": "9.5259" + }, + { + "field": "p99", + "value": "22.8066" + }, + { + "field": "max", + "value": "145.7" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "199" + }, + { + "field": "p50", + "value": "1701.5263" + }, + { + "field": "p90", + "value": "1803.0807" + }, + { + "field": "p99", + "value": "1966.8893" + }, + { + "field": "max", + "value": "2001.37" + } + ] + ], + "statistics": { + "recordsMatched": 141696.0, + "recordsScanned": 1275490.0, + "bytesScanned": 973056498.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.txt new file mode 100644 index 0000000..e09e9bc --- /dev/null +++ b/loadtestcli/Samples/NET8MinimalAPI/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 07:22:52 UTC 2023 +x86 RESULTS lambda: Net8-MinimalApi-X86 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 141497 +RESULTS p50 5.8237 +RESULTS p90 9.5259 +RESULTS p99 22.8066 +RESULTS max 145.7 +RESULTS coldstart 1 +RESULTS count 199 +RESULTS p50 1701.5263 +RESULTS p90 1803.0807 +RESULTS p99 1966.8893 +RESULTS max 2001.37 +STATISTICS 973056498.0 141696.0 1275490.0 diff --git a/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.json b/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.json new file mode 100644 index 0000000..52f202f --- /dev/null +++ b/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.json @@ -0,0 +1,62 @@ +{ + "results": [ + [ + { + "field": "coldstart", + "value": "0" + }, + { + "field": "count", + "value": "136757" + }, + { + "field": "p50", + "value": "5.7719" + }, + { + "field": "p90", + "value": "8.735" + }, + { + "field": "p99", + "value": "19.5334" + }, + { + "field": "max", + "value": "130.47" + } + ], + [ + { + "field": "coldstart", + "value": "1" + }, + { + "field": "count", + "value": "256" + }, + { + "field": "p50", + "value": "365.4209" + }, + { + "field": "p90", + "value": "409.523" + }, + { + "field": "p99", + "value": "463.0947" + }, + { + "field": "max", + "value": "515.67" + } + ] + ], + "statistics": { + "recordsMatched": 137013.0, + "recordsScanned": 516656.0, + "bytesScanned": 171255480.0 + }, + "status": "Complete" +} diff --git a/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.txt new file mode 100644 index 0000000..c08cce0 --- /dev/null +++ b/loadtestcli/Samples/NET8Native/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 07:49:43 UTC 2023 +x86 RESULTS +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 136757 +RESULTS p50 5.7719 +RESULTS p90 8.735 +RESULTS p99 19.5334 +RESULTS max 130.47 +RESULTS coldstart 1 +RESULTS count 256 +RESULTS p50 365.4209 +RESULTS p90 409.523 +RESULTS p99 463.0947 +RESULTS max 515.67 +STATISTICS 171255480.0 137013.0 516656.0 diff --git a/src/NET6MinimalAPI/Report/load-test-report-x86.json b/loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.json similarity index 71% rename from src/NET6MinimalAPI/Report/load-test-report-x86.json rename to loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.json index d2d2920..b991edf 100644 --- a/src/NET6MinimalAPI/Report/load-test-report-x86.json +++ b/loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.json @@ -7,7 +7,7 @@ }, { "field": "count", - "value": "135861" + "value": "141726" }, { "field": "p50", @@ -15,7 +15,7 @@ }, { "field": "p90", - "value": "9.9905" + "value": "10.1504" }, { "field": "p99", @@ -23,7 +23,7 @@ }, { "field": "max", - "value": "108.6" + "value": "209.82" } ], [ @@ -33,30 +33,30 @@ }, { "field": "count", - "value": "197" + "value": "65" }, { "field": "p50", - "value": "1742.8361" + "value": "543.61" }, { "field": "p90", - "value": "1966.8893" + "value": "592.98" }, { "field": "p99", - "value": "2411.7468" + "value": "740.61" }, { "field": "max", - "value": "2503.31" + "value": "740.61" } ] ], "statistics": { - "recordsMatched": 136058.0, - "recordsScanned": 1224739.0, - "bytesScanned": 894477163.0 + "recordsMatched": 141791.0, + "recordsScanned": 1188681.0, + "bytesScanned": 880231703.0 }, "status": "Complete" } diff --git a/loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.txt b/loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.txt new file mode 100644 index 0000000..4613c0d --- /dev/null +++ b/loadtestcli/Samples/NET8NativeMinimalAPI/Report/load-test-report-x86.txt @@ -0,0 +1,18 @@ +Mon Nov 13 08:03:28 UTC 2023 +x86 RESULTS lambda: Net8-Native-MinimalApi-X86 +Test duration sec: 600 +Log interval min: 20 +Complete +RESULTS coldstart 0 +RESULTS count 141726 +RESULTS p50 5.9169 +RESULTS p90 10.1504 +RESULTS p99 21.746 +RESULTS max 209.82 +RESULTS coldstart 1 +RESULTS count 65 +RESULTS p50 543.61 +RESULTS p90 592.98 +RESULTS p99 740.61 +RESULTS max 740.61 +STATISTICS 880231703.0 141791.0 1188681.0 diff --git a/src/NET6/DeleteProduct/DeleteProduct.csproj b/src/NET6/DeleteProduct/DeleteProduct.csproj index c22ca94..3fe0a1c 100755 --- a/src/NET6/DeleteProduct/DeleteProduct.csproj +++ b/src/NET6/DeleteProduct/DeleteProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6/GenerateLoadTestResults/DateUtils.cs b/src/NET6/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET6/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET6/GenerateLoadTestResults/Function.cs b/src/NET6/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET6/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET6/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET6/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET6/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET6/GenerateLoadTestResults/QueryResult.cs b/src/NET6/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET6/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET6/GetProduct/GetProduct.csproj b/src/NET6/GetProduct/GetProduct.csproj index c22ca94..3fe0a1c 100755 --- a/src/NET6/GetProduct/GetProduct.csproj +++ b/src/NET6/GetProduct/GetProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6/GetProducts/GetProducts.csproj b/src/NET6/GetProducts/GetProducts.csproj index c22ca94..3fe0a1c 100755 --- a/src/NET6/GetProducts/GetProducts.csproj +++ b/src/NET6/GetProducts/GetProducts.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6/PutProduct/PutProduct.csproj b/src/NET6/PutProduct/PutProduct.csproj index c22ca94..3fe0a1c 100755 --- a/src/NET6/PutProduct/PutProduct.csproj +++ b/src/NET6/PutProduct/PutProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6/Shared/Shared.csproj b/src/NET6/Shared/Shared.csproj index e1fbf59..e3a7858 100755 --- a/src/NET6/Shared/Shared.csproj +++ b/src/NET6/Shared/Shared.csproj @@ -4,8 +4,8 @@ netstandard2.1 - - - + + + diff --git a/src/NET6/deploy.sh b/src/NET6/deploy.sh new file mode 100755 index 0000000..0a05a23 --- /dev/null +++ b/src/NET6/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet6 +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET6/run-loadtest.sh b/src/NET6/run-loadtest.sh new file mode 100755 index 0000000..054f7ac --- /dev/null +++ b/src/NET6/run-loadtest.sh @@ -0,0 +1,182 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + +STACK_NAME=dotnet6 +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA_GETPRODUCTS Name=FunctionName,Value=$LAMBDA_GETPRODUCT Name=FunctionName,Value=$LAMBDA_DELETEPRODUCT Name=FunctionName,Value=$LAMBDA_PUTPRODUCT \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + + cat ./Report/load-test-errors-$1.json + + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA_GETPRODUCTS" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct +RunLoadTest arm64 ApiUrlArm64 LambdaArm64NameGetProducts LambdaArm64NameGetProduct LambdaArm64NameDeleteProduct LambdaArm64NamePutProduct \ No newline at end of file diff --git a/src/NET6/template.yaml b/src/NET6/template.yaml index 02504fb..848cf70 100755 --- a/src/NET6/template.yaml +++ b/src/NET6/template.yaml @@ -1,13 +1,16 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net6-X86 + arm64FunctionNamePrefix: + Type: String + Default: Net6-Arm64 Globals: - HttpApi: - Auth: - EnableIamAuthorizer: true Function: MemorySize: 1024 - Architectures: [!Ref LambdaArchitecture] Runtime: dotnet6 Timeout: 30 Tracing: Active @@ -15,49 +18,39 @@ Globals: Variables: PRODUCT_TABLE_NAME: !Ref Table -Parameters: - LambdaArchitecture: - Type: String - AllowedValues: - - arm64 - - x86_64 - Description: Enter arm64 or x86_64 - - Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] CodeUri: ./GetProducts/ Handler: GetProducts::GetProducts.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET - Auth: - Authorizer: AWS_IAM Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:Scan - Resource: !GetAtt Table.Arn + - DynamoDBReadPolicy: + TableName: + !Ref Table - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] CodeUri: ./GetProduct/ Handler: GetProduct::GetProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET - Auth: - Authorizer: AWS_IAM Policies: - Version: "2012-10-17" Statement: @@ -65,19 +58,19 @@ Resources: Action: dynamodb:GetItem Resource: !GetAtt Table.Arn - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] CodeUri: ./DeleteProduct/ Handler: DeleteProduct::DeleteProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE - Auth: - Authorizer: AWS_IAM Policies: - Version: "2012-10-17" Statement: @@ -87,57 +80,105 @@ Resources: - dynamodb:GetItem Resource: !GetAtt Table.Arn - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] CodeUri: ./PutProduct/ Handler: PutProduct::PutProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT - Auth: - Authorizer: AWS_IAM Policies: - Version: "2012-10-17" Statement: - Effect: Allow Action: dynamodb:PutItem Resource: !GetAtt Table.Arn +#ARM64 + GetProductsFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + Architectures: [arm64] + CodeUri: ./GetProducts/ + Handler: GetProducts::GetProducts.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64 + Method: GET + Policies: + - DynamoDBReadPolicy: + TableName: + !Ref Table - GenerateLoadTestResults: + GetProductFunctionArm64: Type: AWS::Serverless::Function Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + Architectures: [arm64] + CodeUri: ./GetProduct/ + Handler: GetProduct::GetProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /test-results + Path: /arm64/{id} Method: GET - Auth: - Authorizer: AWS_IAM - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-6-base-" - LOAD_TEST_TYPE: "NET 6" - LAMBDA_ARCHITECTURE: !Ref LambdaArchitecture Policies: - Version: "2012-10-17" Statement: - - Sid: AllowStartQueries - Effect: Allow + - Effect: Allow + Action: dynamodb:GetItem + Resource: !GetAtt Table.Arn + + DeleteProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + Architectures: [arm64] + CodeUri: ./DeleteProduct/ + Handler: DeleteProduct::DeleteProduct.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: DELETE + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + - dynamodb:DeleteItem + - dynamodb:GetItem + Resource: !GetAtt Table.Arn + + PutProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] + Architectures: [arm64] + CodeUri: ./PutProduct/ + Handler: PutProduct::PutProduct.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: PUT + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: dynamodb:PutItem + Resource: !GetAtt Table.Arn Table: Type: AWS::DynamoDB::Table @@ -149,10 +190,42 @@ Resources: KeySchema: - AttributeName: id KeyType: HASH - StreamSpecification: - StreamViewType: NEW_AND_OLD_IMAGES Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + + #arm64 + LambdaArm64NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + LambdaArm64NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + LambdaArm64NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + LambdaArm64NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET6Containers/DeleteProduct/DeleteProduct.csproj b/src/NET6Containers/DeleteProduct/DeleteProduct.csproj index c22ca94..3fe0a1c 100644 --- a/src/NET6Containers/DeleteProduct/DeleteProduct.csproj +++ b/src/NET6Containers/DeleteProduct/DeleteProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6Containers/GenerateLoadTestResults/DateUtils.cs b/src/NET6Containers/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET6Containers/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET6Containers/GenerateLoadTestResults/Function.cs b/src/NET6Containers/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET6Containers/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET6Containers/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET6Containers/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET6Containers/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET6Containers/GenerateLoadTestResults/QueryResult.cs b/src/NET6Containers/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET6Containers/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET6Containers/GetProduct/GetProduct.csproj b/src/NET6Containers/GetProduct/GetProduct.csproj index c22ca94..3fe0a1c 100644 --- a/src/NET6Containers/GetProduct/GetProduct.csproj +++ b/src/NET6Containers/GetProduct/GetProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6Containers/GetProducts/GetProducts.csproj b/src/NET6Containers/GetProducts/GetProducts.csproj index c22ca94..3fe0a1c 100644 --- a/src/NET6Containers/GetProducts/GetProducts.csproj +++ b/src/NET6Containers/GetProducts/GetProducts.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6Containers/PutProduct/PutProduct.csproj b/src/NET6Containers/PutProduct/PutProduct.csproj index c22ca94..3fe0a1c 100644 --- a/src/NET6Containers/PutProduct/PutProduct.csproj +++ b/src/NET6Containers/PutProduct/PutProduct.csproj @@ -6,13 +6,13 @@ true - - - - - - - + + + + + + + diff --git a/src/NET6Containers/Shared/Shared.csproj b/src/NET6Containers/Shared/Shared.csproj index e1fbf59..e3a7858 100644 --- a/src/NET6Containers/Shared/Shared.csproj +++ b/src/NET6Containers/Shared/Shared.csproj @@ -4,8 +4,8 @@ netstandard2.1 - - - + + + diff --git a/src/NET6Containers/deploy.sh b/src/NET6Containers/deploy.sh new file mode 100755 index 0000000..d7e806b --- /dev/null +++ b/src/NET6Containers/deploy.sh @@ -0,0 +1,51 @@ +#Arguments: +#$1 - ECR URI +#$2 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet6-container +ECR_URI=NotSet +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + ECR_URI=$1 +fi + +if [ "x$2" != x ]; +then + DELETE_STACK=$2 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo ECR URI: $ECR_URI +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --image-repository $ECR_URI --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET6Containers/run-loadtest.sh b/src/NET6Containers/run-loadtest.sh new file mode 100755 index 0000000..3fd4dbd --- /dev/null +++ b/src/NET6Containers/run-loadtest.sh @@ -0,0 +1,180 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + +STACK_NAME=dotnet6-container +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA_GETPRODUCTS Name=FunctionName,Value=$LAMBDA_GETPRODUCT Name=FunctionName,Value=$LAMBDA_DELETEPRODUCT Name=FunctionName,Value=$LAMBDA_PUTPRODUCT \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + + cat ./Report/load-test-errors-$1.json + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA_GETPRODUCTS" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct \ No newline at end of file diff --git a/src/NET6Containers/template.yaml b/src/NET6Containers/template.yaml index f8c0a29..b9c6ff1 100644 --- a/src/NET6Containers/template.yaml +++ b/src/NET6Containers/template.yaml @@ -1,42 +1,37 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net6-Container-X86 Globals: Function: MemorySize: 1024 - Architectures: [!Ref LambdaArchitecture] - Runtime: dotnet6 Timeout: 30 Tracing: Active Environment: Variables: PRODUCT_TABLE_NAME: !Ref Table -Parameters: - LambdaArchitecture: - Type: String - AllowedValues: - - arm64 - - x86_64 - Description: Enter arm64 or x86_64 - Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] PackageType: Image Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:Scan - Resource: !GetAtt Table.Arn + - DynamoDBReadPolicy: + TableName: + !Ref Table Metadata: DockerTag: dotnet6-v1 DockerContext: ../../ @@ -44,15 +39,17 @@ Resources: DockerBuildArgs: SAM_BUILD_MODE: run - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: PackageType: Image + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -67,15 +64,17 @@ Resources: DockerBuildArgs: SAM_BUILD_MODE: run - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] PackageType: Image Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -92,15 +91,17 @@ Resources: DockerBuildArgs: SAM_BUILD_MODE: run - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] PackageType: Image Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -115,36 +116,6 @@ Resources: DockerBuildArgs: SAM_BUILD_MODE: run - GenerateLoadTestResults: - Type: AWS::Serverless::Function - Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler - Events: - Api: - Type: HttpApi - Properties: - Path: /test-results - Method: GET - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-6-containers-" - LOAD_TEST_TYPE: "NET 6 Containers" - LAMBDA_ARCHITECTURE: !Ref LambdaArchitecture - Policies: - - Version: "2012-10-17" - Statement: - - Sid: AllowStartQueries - Effect: Allow - Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" - Table: Type: AWS::DynamoDB::Table Properties: @@ -155,10 +126,25 @@ Resources: KeySchema: - AttributeName: id KeyType: HASH - StreamSpecification: - StreamViewType: NEW_AND_OLD_IMAGES Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET6CustomRuntime/DeleteProduct/DeleteProduct.csproj b/src/NET6CustomRuntime/DeleteProduct/DeleteProduct.csproj index 7f2069d..f733a6a 100644 --- a/src/NET6CustomRuntime/DeleteProduct/DeleteProduct.csproj +++ b/src/NET6CustomRuntime/DeleteProduct/DeleteProduct.csproj @@ -15,12 +15,12 @@ - - - - - - + + + + + + diff --git a/src/NET6CustomRuntime/DeleteProduct/aws-lambda-tools-defaults.json b/src/NET6CustomRuntime/DeleteProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET6CustomRuntime/DeleteProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6CustomRuntime/GenerateLoadTestResults/DateUtils.cs b/src/NET6CustomRuntime/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET6CustomRuntime/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET6CustomRuntime/GenerateLoadTestResults/Function.cs b/src/NET6CustomRuntime/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET6CustomRuntime/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET6CustomRuntime/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET6CustomRuntime/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET6CustomRuntime/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET6CustomRuntime/GenerateLoadTestResults/QueryResult.cs b/src/NET6CustomRuntime/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET6CustomRuntime/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET6CustomRuntime/GetProduct/GetProduct.csproj b/src/NET6CustomRuntime/GetProduct/GetProduct.csproj index e94ebb7..355ae20 100644 --- a/src/NET6CustomRuntime/GetProduct/GetProduct.csproj +++ b/src/NET6CustomRuntime/GetProduct/GetProduct.csproj @@ -13,12 +13,12 @@ linux-x64 - - - - - - + + + + + + diff --git a/src/NET6CustomRuntime/GetProduct/aws-lambda-tools-defaults.json b/src/NET6CustomRuntime/GetProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET6CustomRuntime/GetProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6CustomRuntime/GetProducts/GetProducts.csproj b/src/NET6CustomRuntime/GetProducts/GetProducts.csproj index 11ab6ab..ef13d3b 100644 --- a/src/NET6CustomRuntime/GetProducts/GetProducts.csproj +++ b/src/NET6CustomRuntime/GetProducts/GetProducts.csproj @@ -14,12 +14,12 @@ linux-x64 - - - - - - + + + + + + diff --git a/src/NET6CustomRuntime/GetProducts/aws-lambda-tools-defaults.json b/src/NET6CustomRuntime/GetProducts/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET6CustomRuntime/GetProducts/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6CustomRuntime/Makefile b/src/NET6CustomRuntime/Makefile index 2aadd31..4850513 100644 --- a/src/NET6CustomRuntime/Makefile +++ b/src/NET6CustomRuntime/Makefile @@ -1,15 +1,23 @@ -build-GetProductFunction: - dotnet clean +build-GetProductFunctionX86: dotnet publish GetProduct/GetProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-GetProductsFunction: - dotnet clean +build-GetProductsFunctionX86: dotnet publish GetProducts/GetProducts.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-PutProductFunction: - dotnet clean +build-PutProductFunctionX86: dotnet publish PutProduct/PutProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-DeleteProductFunction: - dotnet clean - dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file +build-DeleteProductFunctionX86: + dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) + +build-GetProductFunctionArm64: + dotnet publish GetProduct/GetProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-GetProductsFunctionArm64: + dotnet publish GetProducts/GetProducts.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-PutProductFunctionArm64: + dotnet publish PutProduct/PutProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-DeleteProductFunctionArm64: + dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file diff --git a/src/NET6CustomRuntime/PutProduct/PutProduct.csproj b/src/NET6CustomRuntime/PutProduct/PutProduct.csproj index fcfcccc..f3ddff1 100644 --- a/src/NET6CustomRuntime/PutProduct/PutProduct.csproj +++ b/src/NET6CustomRuntime/PutProduct/PutProduct.csproj @@ -15,12 +15,12 @@ - - - - - - + + + + + + diff --git a/src/NET6CustomRuntime/PutProduct/aws-lambda-tools-defaults.json b/src/NET6CustomRuntime/PutProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 7ed1688..0000000 --- a/src/NET6CustomRuntime/PutProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "Information": [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - "dotnet lambda help", - "All the command line options for the Lambda command can be specified in this file." - ], - "profile": "", - "region": "", - "configuration": "Release", - "function-runtime": "provided.al2", - "function-memory-size": 256, - "function-timeout": 30, - "function-handler": "bootstrap" -} \ No newline at end of file diff --git a/src/NET6CustomRuntime/Shared/Shared.csproj b/src/NET6CustomRuntime/Shared/Shared.csproj index 3416950..b8db67f 100644 --- a/src/NET6CustomRuntime/Shared/Shared.csproj +++ b/src/NET6CustomRuntime/Shared/Shared.csproj @@ -13,10 +13,10 @@ linux-x64 - - - - + + + + diff --git a/src/NET6CustomRuntime/deploy.sh b/src/NET6CustomRuntime/deploy.sh new file mode 100755 index 0000000..a87f3e6 --- /dev/null +++ b/src/NET6CustomRuntime/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet6-custom +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET6CustomRuntime/run-loadtest.sh b/src/NET6CustomRuntime/run-loadtest.sh new file mode 100755 index 0000000..35d335d --- /dev/null +++ b/src/NET6CustomRuntime/run-loadtest.sh @@ -0,0 +1,181 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + +STACK_NAME=dotnet6-custom +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA_GETPRODUCTS Name=FunctionName,Value=$LAMBDA_GETPRODUCT Name=FunctionName,Value=$LAMBDA_DELETEPRODUCT Name=FunctionName,Value=$LAMBDA_PUTPRODUCT \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + + cat ./Report/load-test-errors-$1.json + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA_GETPRODUCTS" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct +RunLoadTest arm64 ApiUrlArm64 LambdaArm64NameGetProducts LambdaArm64NameGetProduct LambdaArm64NameDeleteProduct LambdaArm64NamePutProduct \ No newline at end of file diff --git a/src/NET6CustomRuntime/template.yaml b/src/NET6CustomRuntime/template.yaml index c6c34f7..477c3ad 100644 --- a/src/NET6CustomRuntime/template.yaml +++ b/src/NET6CustomRuntime/template.yaml @@ -1,10 +1,17 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net6-X86-Custom + arm64FunctionNamePrefix: + Type: String + Default: Net6-Arm64-Custom + Globals: Function: MemorySize: 1024 - Architectures: ["x86_64"] Runtime: provided.al2 Timeout: 30 Tracing: Active @@ -13,16 +20,19 @@ Globals: PRODUCT_TABLE_NAME: !Ref Table Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] CodeUri: ./ Handler: bootstrap Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - DynamoDBReadPolicy: @@ -31,16 +41,18 @@ Resources: Metadata: BuildMethod: makefile - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -51,16 +63,18 @@ Resources: Metadata: BuildMethod: makefile - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: DeleteProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -73,16 +87,18 @@ Resources: Metadata: BuildMethod: makefile - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: PutProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -92,37 +108,94 @@ Resources: Resource: !GetAtt Table.Arn Metadata: BuildMethod: makefile + #ARM64 + GetProductsFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + Architectures: [arm64] + CodeUri: ./ + Handler: bootstrap + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64 + Method: GET + Policies: + - DynamoDBReadPolicy: + TableName: + !Ref Table + Metadata: + BuildMethod: makefile - GenerateLoadTestResults: + GetProductFunctionArm64: Type: AWS::Serverless::Function Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler - Runtime: dotnet6 + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /test-results + Path: /arm64/{id} Method: GET - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-8-base-" - LOAD_TEST_TYPE: "NET 8" - LAMBDA_ARCHITECTURE: "x86_64" Policies: - Version: "2012-10-17" Statement: - - Sid: AllowStartQueries - Effect: Allow + - Effect: Allow + Action: dynamodb:GetItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile + + DeleteProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: DeleteProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: DELETE + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + - dynamodb:DeleteItem + - dynamodb:GetItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile + + PutProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: PutProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: PUT + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: dynamodb:PutItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile Table: Type: AWS::DynamoDB::Table @@ -138,4 +211,38 @@ Resources: Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + + #arm64 + LambdaArm64NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + LambdaArm64NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + LambdaArm64NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + LambdaArm64NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET6MinimalAPI/Report/load-test-report-arm64.txt b/src/NET6MinimalAPI/Report/load-test-report-arm64.txt deleted file mode 100644 index 4ae278b..0000000 --- a/src/NET6MinimalAPI/Report/load-test-report-arm64.txt +++ /dev/null @@ -1,18 +0,0 @@ -Sun Nov 5 18:51:08 UTC 2023 -arm64 RESULTS lambda: Net6-MinimalApi-Arm64 -Test duration sec: 600 -Log interval min: 20 -Complete -RESULTS coldstart 0 -RESULTS count 135963 -RESULTS p50 6.2055 -RESULTS p90 9.6783 -RESULTS p99 20.0868 -RESULTS max 528.13 -RESULTS coldstart 1 -RESULTS count 246 -RESULTS p50 2105.2185 -RESULTS p90 2164.9673 -RESULTS p99 2215.3132 -RESULTS max 2228.18 -STATISTICS 900312482.0 136209.0 1226110.0 diff --git a/src/NET6MinimalAPI/Report/load-test-report-x86.txt b/src/NET6MinimalAPI/Report/load-test-report-x86.txt deleted file mode 100644 index 21b5798..0000000 --- a/src/NET6MinimalAPI/Report/load-test-report-x86.txt +++ /dev/null @@ -1,18 +0,0 @@ -Sun Nov 5 18:40:27 UTC 2023 -x86 RESULTS lambda: Net6-MinimalApi-X86 -Test duration sec: 600 -Log interval min: 20 -Complete -RESULTS coldstart 0 -RESULTS count 135861 -RESULTS p50 5.9169 -RESULTS p90 9.9905 -RESULTS p99 21.746 -RESULTS max 108.6 -RESULTS coldstart 1 -RESULTS count 197 -RESULTS p50 1742.8361 -RESULTS p90 1966.8893 -RESULTS p99 2411.7468 -RESULTS max 2503.31 -STATISTICS 894477163.0 136058.0 1224739.0 diff --git a/src/NET6MinimalAPI/deploy.sh b/src/NET6MinimalAPI/deploy.sh index 6876c29..7897959 100755 --- a/src/NET6MinimalAPI/deploy.sh +++ b/src/NET6MinimalAPI/deploy.sh @@ -7,7 +7,7 @@ DELETE_STACK=yes COLOR='\033[0;33m' NO_COLOR='\033[0m' # No Color -if [ "x$4" != x ]; +if [ "x$1" != x ]; then DELETE_STACK=$1 fi diff --git a/src/NET6MinimalAPI/run-loadtest.sh b/src/NET6MinimalAPI/run-loadtest.sh index 0832846..ca2a1c3 100755 --- a/src/NET6MinimalAPI/run-loadtest.sh +++ b/src/NET6MinimalAPI/run-loadtest.sh @@ -1,7 +1,8 @@ #Arguments: #$1 - load test duration in seconds #$2 - log interval to be used in the cloudwatch query in minutes -#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results STACK_NAME=dotnet6-minimal-api TEST_DURATIOMN_SEC=60 @@ -26,15 +27,21 @@ then LOG_DELETE=$3 fi +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + echo "${COLOR}" echo -------------------------------------------- echo DURATION:$TEST_DURATIOMN_SEC echo LOG INTERVAL:$LOG_INTERVAL_MIN echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN echo -------------------------------------------- echo "${NO_COLOR}" -mkdir -p Report +mkdir -p Report function RunLoadTest() { @@ -45,7 +52,7 @@ function RunLoadTest() #get test params from cloud formation output echo "${COLOR}" - API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ --output text) echo API URL: $API_URL @@ -73,10 +80,9 @@ function RunLoadTest() echo -------------------------------------------- echo "${NO_COLOR}" artillery run \ - --target "$API_URL" \ --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ --quiet \ - ../../loadtest/load-test.yml + ../../loadtest/codebuild/load-test.yml echo "${COLOR}" echo -------------------------------------------- @@ -91,6 +97,16 @@ function RunLoadTest() echo Log start:$startdate end:$enddate echo -------------------------------------------- + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + QUERY_ID=$(aws logs start-query \ --log-group-name /aws/lambda/$LAMBDA \ --start-time $startdate \ @@ -118,6 +134,16 @@ function RunLoadTest() aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt cat ./Report/load-test-report-$1.txt aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi } RunLoadTest x86 ApiUrlX86 LambdaX86Name diff --git a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt b/src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt deleted file mode 100644 index 6df5dbc..0000000 --- a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-arm64.txt +++ /dev/null @@ -1,18 +0,0 @@ -Sun Nov 5 19:14:52 UTC 2023 -arm64 RESULTS lambda: Net6-MinimalApi-WebadApter-Arm64 -Test duration sec: 600 -Log interval min: 20 -Complete -RESULTS coldstart 0 -RESULTS count 138527 -RESULTS p50 6.1078 -RESULTS p90 9.3759 -RESULTS p99 17.9744 -RESULTS max 838.78 -RESULTS coldstart 1 -RESULTS count 139 -RESULTS p50 1277.1986 -RESULTS p90 1326.6409 -RESULTS p99 1358.8491 -RESULTS max 1367.49 -STATISTICS 268615241.0 138666.0 694392.0 diff --git a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt b/src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt deleted file mode 100644 index 1f5f629..0000000 --- a/src/NET6MinimalAPIWebAdapter/Report/load-test-report-x86.txt +++ /dev/null @@ -1,18 +0,0 @@ -Sun Nov 5 19:04:10 UTC 2023 -x86 RESULTS lambda: Net6-MinimalApi-WebadApter-X86 -Test duration sec: 600 -Log interval min: 20 -Complete -RESULTS coldstart 0 -RESULTS count 140872 -RESULTS p50 6.2055 -RESULTS p90 10.3128 -RESULTS p99 21.746 -RESULTS max 154.62 -RESULTS coldstart 1 -RESULTS count 112 -RESULTS p50 1013.88 -RESULTS p90 1102.6789 -RESULTS p99 1330.6248 -RESULTS max 1392.85 -STATISTICS 272986184.0 140984.0 705687.0 diff --git a/src/NET6MinimalAPIWebAdapter/deploy.sh b/src/NET6MinimalAPIWebAdapter/deploy.sh index 1cff98f..5c4dade 100755 --- a/src/NET6MinimalAPIWebAdapter/deploy.sh +++ b/src/NET6MinimalAPIWebAdapter/deploy.sh @@ -7,7 +7,7 @@ DELETE_STACK=yes COLOR='\033[0;33m' NO_COLOR='\033[0m' # No Color -if [ "x$4" != x ]; +if [ "x$1" != x ]; then DELETE_STACK=$1 fi diff --git a/src/NET6MinimalAPIWebAdapter/run-loadtest.sh b/src/NET6MinimalAPIWebAdapter/run-loadtest.sh index 003446e..2242a7f 100755 --- a/src/NET6MinimalAPIWebAdapter/run-loadtest.sh +++ b/src/NET6MinimalAPIWebAdapter/run-loadtest.sh @@ -1,7 +1,9 @@ #Arguments: #$1 - load test duration in seconds #$2 - log interval to be used in the cloudwatch query in minutes -#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + STACK_NAME=dotnet6-minimal-api-web-adapter TEST_DURATIOMN_SEC=60 LOG_INTERVAL_MIN=20 @@ -25,11 +27,17 @@ then LOG_DELETE=$3 fi +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + echo "${COLOR}" echo -------------------------------------------- echo DURATION:$TEST_DURATIOMN_SEC echo LOG INTERVAL:$LOG_INTERVAL_MIN echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN echo -------------------------------------------- echo "${NO_COLOR}" @@ -44,7 +52,7 @@ function RunLoadTest() #get test params from cloud formation output echo "${COLOR}" - API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ --output text) echo API URL: $API_URL @@ -72,10 +80,9 @@ function RunLoadTest() echo -------------------------------------------- echo "${NO_COLOR}" artillery run \ - --target "$API_URL" \ --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ --quiet \ - ../../loadtest/load-test.yml + ../../loadtest/codebuild/load-test.yml echo "${COLOR}" echo -------------------------------------------- @@ -90,6 +97,16 @@ function RunLoadTest() echo Log start:$startdate end:$enddate echo -------------------------------------------- + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + QUERY_ID=$(aws logs start-query \ --log-group-name /aws/lambda/$LAMBDA \ --start-time $startdate \ @@ -117,6 +134,16 @@ function RunLoadTest() aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt cat ./Report/load-test-report-$1.txt aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi } RunLoadTest x86 ApiUrlX86 LambdaX86Name diff --git a/src/NET6Native/DeleteProduct/DeleteProduct.csproj b/src/NET6Native/DeleteProduct/DeleteProduct.csproj deleted file mode 100755 index db7b4ad..0000000 --- a/src/NET6Native/DeleteProduct/DeleteProduct.csproj +++ /dev/null @@ -1,23 +0,0 @@ - - - - exe - net6.0 - true - true - - - - - - - - - - - - - - - - diff --git a/src/NET6Native/DeleteProduct/Function.cs b/src/NET6Native/DeleteProduct/Function.cs deleted file mode 100755 index 92c0989..0000000 --- a/src/NET6Native/DeleteProduct/Function.cs +++ /dev/null @@ -1,73 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Net; -using System.Net.Http; -using System.Text.Json.Serialization; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; -using Amazon.Lambda.RuntimeSupport; -using Amazon.Lambda.Serialization.SystemTextJson; -using Amazon.XRay.Recorder.Handlers.AwsSdk; -using Shared.DataAccess; - -[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer))] - -AWSSDKHandler.RegisterXRayForAllServices(); -ProductsDAO dataAccess = new DynamoDbProducts(); - -var handler = async (APIGatewayHttpApiV2ProxyRequest apigProxyEvent, ILambdaContext context) => -{ - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Delete.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only DELETE allowed", - StatusCode = (int)HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var id = apigProxyEvent.PathParameters["id"]; - - var product = await dataAccess.GetProduct(id); - - if (product == null) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.NotFound, - }; - } - - await dataAccess.DeleteProduct(product.Id); - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int)HttpStatusCode.OK, - Body = $"Product with id {id} deleted" - }; - } - catch (Exception e) - { - context.Logger.LogError($"Error deleting product {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.InternalServerError, - }; - } -}; - -await LambdaBootstrapBuilder.Create(handler, new DefaultLambdaJsonSerializer()) - .Build() - .RunAsync(); - -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] -[JsonSerializable(typeof(Dictionary))] -public partial class ApiGatewayProxyJsonSerializerContext : JsonSerializerContext -{ -} \ No newline at end of file diff --git a/src/NET6Native/DeleteProduct/aws-lambda-tools-defaults.json b/src/NET6Native/DeleteProduct/aws-lambda-tools-defaults.json deleted file mode 100755 index 1d112e0..0000000 --- a/src/NET6Native/DeleteProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6Native/GetProduct/Function.cs b/src/NET6Native/GetProduct/Function.cs deleted file mode 100755 index 7d5e3e1..0000000 --- a/src/NET6Native/GetProduct/Function.cs +++ /dev/null @@ -1,73 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Text.Json.Serialization; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; -using Amazon.Lambda.RuntimeSupport; -using Amazon.Lambda.Serialization.SystemTextJson; -using Amazon.XRay.Recorder.Handlers.AwsSdk; -using Shared.DataAccess; - -[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer))] - -AWSSDKHandler.RegisterXRayForAllServices(); -ProductsDAO dataAccess = new DynamoDbProducts(); - -var handler = async (APIGatewayHttpApiV2ProxyRequest apigProxyEvent, ILambdaContext context) => -{ - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int)HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var id = apigProxyEvent.PathParameters["id"]; - - var product = await dataAccess.GetProduct(id); - - if (product == null) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.NotFound, - }; - } - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int)HttpStatusCode.OK, - Body = JsonSerializer.Serialize(product), - Headers = new Dictionary {{"Content-Type", "application/json"}} - }; - } - catch (Exception e) - { - context.Logger.LogError($"Error getting product {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.InternalServerError, - }; - } -}; - -await LambdaBootstrapBuilder.Create(handler, new DefaultLambdaJsonSerializer()) - .Build() - .RunAsync(); - -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] -[JsonSerializable(typeof(Dictionary))] -public partial class ApiGatewayProxyJsonSerializerContext : JsonSerializerContext -{ -} diff --git a/src/NET6Native/GetProduct/GetProduct.csproj b/src/NET6Native/GetProduct/GetProduct.csproj deleted file mode 100755 index db7b4ad..0000000 --- a/src/NET6Native/GetProduct/GetProduct.csproj +++ /dev/null @@ -1,23 +0,0 @@ - - - - exe - net6.0 - true - true - - - - - - - - - - - - - - - - diff --git a/src/NET6Native/GetProduct/aws-lambda-tools-defaults.json b/src/NET6Native/GetProduct/aws-lambda-tools-defaults.json deleted file mode 100755 index 1d112e0..0000000 --- a/src/NET6Native/GetProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6Native/GetProducts/Function.cs b/src/NET6Native/GetProducts/Function.cs deleted file mode 100755 index f19d6f1..0000000 --- a/src/NET6Native/GetProducts/Function.cs +++ /dev/null @@ -1,53 +0,0 @@ -using System.Collections.Generic; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Text.Json.Serialization; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; -using Amazon.Lambda.RuntimeSupport; -using Amazon.Lambda.Serialization.SystemTextJson; -using Amazon.XRay.Recorder.Handlers.AwsSdk; -using Shared.DataAccess; - -[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer))] - -AWSSDKHandler.RegisterXRayForAllServices(); -ProductsDAO dataAccess = new DynamoDbProducts(); - -var handler = async (APIGatewayHttpApiV2ProxyRequest apigProxyEvent, ILambdaContext context) => -{ - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int)HttpStatusCode.MethodNotAllowed, - }; - } - - context.Logger.LogInformation($"Received {apigProxyEvent}"); - - var products = await dataAccess.GetAllProducts(); - - context.Logger.LogInformation($"Found {products.Products.Count} product(s)"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = JsonSerializer.Serialize(products), - StatusCode = 200, - Headers = new Dictionary {{"Content-Type", "application/json"}} - }; -}; - -await LambdaBootstrapBuilder.Create(handler, new DefaultLambdaJsonSerializer()) - .Build() - .RunAsync(); - -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] -[JsonSerializable(typeof(List))] -[JsonSerializable(typeof(Dictionary))] -public partial class ApiGatewayProxyJsonSerializerContext : JsonSerializerContext -{ -} diff --git a/src/NET6Native/GetProducts/GetProducts.csproj b/src/NET6Native/GetProducts/GetProducts.csproj deleted file mode 100755 index db7b4ad..0000000 --- a/src/NET6Native/GetProducts/GetProducts.csproj +++ /dev/null @@ -1,23 +0,0 @@ - - - - exe - net6.0 - true - true - - - - - - - - - - - - - - - - diff --git a/src/NET6Native/GetProducts/aws-lambda-tools-defaults.json b/src/NET6Native/GetProducts/aws-lambda-tools-defaults.json deleted file mode 100755 index 1d112e0..0000000 --- a/src/NET6Native/GetProducts/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6Native/PutProduct/Function.cs b/src/NET6Native/PutProduct/Function.cs deleted file mode 100755 index 85b2341..0000000 --- a/src/NET6Native/PutProduct/Function.cs +++ /dev/null @@ -1,75 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Text.Json.Serialization; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; -using Amazon.Lambda.RuntimeSupport; -using Amazon.Lambda.Serialization.SystemTextJson; -using Amazon.XRay.Recorder.Handlers.AwsSdk; -using Shared.DataAccess; -using Shared.Models; - -[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer))] - -AWSSDKHandler.RegisterXRayForAllServices(); -ProductsDAO dataAccess = new DynamoDbProducts(); - -var handler = async (APIGatewayHttpApiV2ProxyRequest apigProxyEvent, ILambdaContext context) => -{ - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Put.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only PUT allowed", - StatusCode = (int)HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var id = apigProxyEvent.PathParameters["id"]; - - var product = JsonSerializer.Deserialize(apigProxyEvent.Body); - - if (product == null || id != product.Id) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Product ID in the body does not match path parameter", - StatusCode = (int)HttpStatusCode.BadRequest, - }; - } - - await dataAccess.PutProduct(product); - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int)HttpStatusCode.Created, - Body = $"Created product with id {id}" - }; - } - catch (Exception e) - { - context.Logger.LogError($"Error creating product {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.InternalServerError, - }; - } -}; - -await LambdaBootstrapBuilder.Create(handler, new DefaultLambdaJsonSerializer()) - .Build() - .RunAsync(); - -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))] -[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))] -[JsonSerializable(typeof(Dictionary))] -public partial class ApiGatewayProxyJsonSerializerContext : JsonSerializerContext -{ -} diff --git a/src/NET6Native/PutProduct/PutProduct.csproj b/src/NET6Native/PutProduct/PutProduct.csproj deleted file mode 100755 index db7b4ad..0000000 --- a/src/NET6Native/PutProduct/PutProduct.csproj +++ /dev/null @@ -1,23 +0,0 @@ - - - - exe - net6.0 - true - true - - - - - - - - - - - - - - - - diff --git a/src/NET6Native/PutProduct/aws-lambda-tools-defaults.json b/src/NET6Native/PutProduct/aws-lambda-tools-defaults.json deleted file mode 100755 index 1d112e0..0000000 --- a/src/NET6Native/PutProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET6Native/Shared/DataAccess/DynamoDbProducts.cs b/src/NET6Native/Shared/DataAccess/DynamoDbProducts.cs deleted file mode 100755 index 9feb6e2..0000000 --- a/src/NET6Native/Shared/DataAccess/DynamoDbProducts.cs +++ /dev/null @@ -1,62 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Threading.Tasks; -using Amazon.DynamoDBv2; -using Amazon.DynamoDBv2.Model; -using Shared.Models; - -namespace Shared.DataAccess -{ - public class DynamoDbProducts : ProductsDAO - { - private static readonly string PRODUCT_TABLE_NAME = Environment.GetEnvironmentVariable("PRODUCT_TABLE_NAME") ?? string.Empty; - private readonly AmazonDynamoDBClient _dynamoDbClient; - - public DynamoDbProducts() - { - this._dynamoDbClient = new AmazonDynamoDBClient(); - } - - public async Task GetProduct(string id) - { - var getItemResponse = await this._dynamoDbClient.GetItemAsync(new GetItemRequest(PRODUCT_TABLE_NAME, - new Dictionary(1) - { - {ProductMapper.PK, new AttributeValue(id)} - })); - - return getItemResponse.IsItemSet ? ProductMapper.ProductFromDynamoDB(getItemResponse.Item) : null; - } - - public async Task PutProduct(Product product) - { - await this._dynamoDbClient.PutItemAsync(PRODUCT_TABLE_NAME, ProductMapper.ProductToDynamoDb(product)); - } - - public async Task DeleteProduct(string id) - { - await this._dynamoDbClient.DeleteItemAsync(PRODUCT_TABLE_NAME, new Dictionary(1) - { - {ProductMapper.PK, new AttributeValue(id)} - }); - } - - public async Task GetAllProducts() - { - var data = await this._dynamoDbClient.ScanAsync(new ScanRequest() - { - TableName = PRODUCT_TABLE_NAME, - Limit = 20 - }); - - var products = new List(); - - foreach (var item in data.Items) - { - products.Add(ProductMapper.ProductFromDynamoDB(item)); - } - - return new ProductWrapper(products); - } - } -} \ No newline at end of file diff --git a/src/NET6Native/Shared/DataAccess/ProductMapper.cs b/src/NET6Native/Shared/DataAccess/ProductMapper.cs deleted file mode 100755 index 8e073ed..0000000 --- a/src/NET6Native/Shared/DataAccess/ProductMapper.cs +++ /dev/null @@ -1,33 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Globalization; -using Amazon.DynamoDBv2.Model; -using Shared.Models; - -namespace Shared.DataAccess -{ - public class ProductMapper - { - public static string PK = "id"; - public static string NAME = "name"; - public static string PRICE = "price"; - - public static Product ProductFromDynamoDB(Dictionary items) { - var product = new Product(items[PK].S, items[NAME].S, decimal.Parse(items[PRICE].N)); - - return product; - } - - public static Dictionary ProductToDynamoDb(Product product) { - Dictionary item = new Dictionary(3); - item.Add(PK, new AttributeValue(product.Id)); - item.Add(NAME, new AttributeValue(product.Name)); - item.Add(PRICE, new AttributeValue() - { - N = product.Price.ToString(CultureInfo.InvariantCulture) - }); - - return item; - } - } -} \ No newline at end of file diff --git a/src/NET6Native/Shared/DataAccess/ProductsDAO.cs b/src/NET6Native/Shared/DataAccess/ProductsDAO.cs deleted file mode 100755 index d09decd..0000000 --- a/src/NET6Native/Shared/DataAccess/ProductsDAO.cs +++ /dev/null @@ -1,16 +0,0 @@ -using System.Threading.Tasks; -using Shared.Models; - -namespace Shared.DataAccess -{ - public interface ProductsDAO - { - Task GetProduct(string id); - - Task PutProduct(Product product); - - Task DeleteProduct(string id); - - Task GetAllProducts(); - } -} \ No newline at end of file diff --git a/src/NET6Native/Shared/Models/Product.cs b/src/NET6Native/Shared/Models/Product.cs deleted file mode 100755 index 8264cb7..0000000 --- a/src/NET6Native/Shared/Models/Product.cs +++ /dev/null @@ -1,40 +0,0 @@ -using System; - -namespace Shared.Models -{ - public class Product - { - public Product() - { - this.Id = string.Empty; - this.Name = string.Empty; - } - - public Product(string id, string name, decimal price) - { - this.Id = id; - this.Name = name; - this.Price = price; - } - - public string Id { get; set; } - - public string Name { get; set; } - - public decimal Price { get; private set; } - - public void SetPrice(decimal newPrice) - { - this.Price = Math.Round(newPrice, 2); - } - - public override string ToString() - { - return "Product{" + - "id='" + this.Id + '\'' + - ", name='" + this.Name + '\'' + - ", price=" + this.Price + - '}'; - } - } -} \ No newline at end of file diff --git a/src/NET6Native/Shared/Models/ProductWrapper.cs b/src/NET6Native/Shared/Models/ProductWrapper.cs deleted file mode 100755 index dab4592..0000000 --- a/src/NET6Native/Shared/Models/ProductWrapper.cs +++ /dev/null @@ -1,19 +0,0 @@ -using System.Collections.Generic; - -namespace Shared.Models -{ - public class ProductWrapper - { - public ProductWrapper() - { - this.Products = new List(); - } - - public ProductWrapper(List products) - { - this.Products = products; - } - - public List Products { get; set; } - } -} \ No newline at end of file diff --git a/src/NET6Native/Shared/Shared.csproj b/src/NET6Native/Shared/Shared.csproj deleted file mode 100755 index 9fce83c..0000000 --- a/src/NET6Native/Shared/Shared.csproj +++ /dev/null @@ -1,15 +0,0 @@ - - - - net6.0 - enable - enable - - - - - - - - - diff --git a/src/NET6Native/template.yaml b/src/NET6Native/template.yaml deleted file mode 100755 index dffeb21..0000000 --- a/src/NET6Native/template.yaml +++ /dev/null @@ -1,106 +0,0 @@ -AWSTemplateFormatVersion: "2010-09-09" -Transform: AWS::Serverless-2016-10-31 - -Globals: - Function: - MemorySize: 1024 - Architectures: ["arm64"] - Runtime: provided.al2 - Timeout: 5 - Tracing: Active - Environment: - Variables: - PRODUCT_TABLE_NAME: !Ref Table - -Resources: - GetProductsFunction: - Type: AWS::Serverless::Function - Properties: - CodeUri: ./GetProducts/ - Handler: GetProducts - Events: - Api: - Type: HttpApi - Properties: - Path: / - Method: GET - Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:Scan - Resource: !GetAtt Table.Arn - - GetProductFunction: - Type: AWS::Serverless::Function - Properties: - CodeUri: ./GetProduct/ - Handler: GetProduct - Events: - Api: - Type: HttpApi - Properties: - Path: /{id} - Method: GET - Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:GetItem - Resource: !GetAtt Table.Arn - - DeleteProductFunction: - Type: AWS::Serverless::Function - Properties: - CodeUri: ./DeleteProduct/ - Handler: DeleteProduct - Events: - Api: - Type: HttpApi - Properties: - Path: /{id} - Method: DELETE - Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: - - dynamodb:DeleteItem - - dynamodb:GetItem - Resource: !GetAtt Table.Arn - - PutProductFunction: - Type: AWS::Serverless::Function - Properties: - CodeUri: ./PutProduct/ - Handler: PutProduct - Events: - Api: - Type: HttpApi - Properties: - Path: /{id} - Method: PUT - Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:PutItem - Resource: !GetAtt Table.Arn - - Table: - Type: AWS::DynamoDB::Table - Properties: - AttributeDefinitions: - - AttributeName: id - AttributeType: S - BillingMode: PAY_PER_REQUEST - KeySchema: - - AttributeName: id - KeyType: HASH - StreamSpecification: - StreamViewType: NEW_AND_OLD_IMAGES - -Outputs: - ApiUrl: - Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file diff --git a/src/NET6TopLevelStatements/DeleteProduct/DeleteProduct.csproj b/src/NET6TopLevelStatements/DeleteProduct/DeleteProduct.csproj index ba213d4..8b25d8b 100755 --- a/src/NET6TopLevelStatements/DeleteProduct/DeleteProduct.csproj +++ b/src/NET6TopLevelStatements/DeleteProduct/DeleteProduct.csproj @@ -7,14 +7,14 @@ true - - - - - - - - + + + + + + + + diff --git a/src/NET6TopLevelStatements/GenerateLoadTestResults/DateUtils.cs b/src/NET6TopLevelStatements/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET6TopLevelStatements/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET6TopLevelStatements/GenerateLoadTestResults/Function.cs b/src/NET6TopLevelStatements/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 0b701ce..0000000 --- a/src/NET6TopLevelStatements/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,125 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler(APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int)HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}".Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - var wrapper = new QueryResultWrapper() - { - LoadTestType = $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int)HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int)HttpStatusCode.InternalServerError, - }; - } - } - } -} diff --git a/src/NET6TopLevelStatements/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET6TopLevelStatements/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET6TopLevelStatements/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET6TopLevelStatements/GenerateLoadTestResults/QueryResult.cs b/src/NET6TopLevelStatements/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET6TopLevelStatements/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET6TopLevelStatements/GetProduct/GetProduct.csproj b/src/NET6TopLevelStatements/GetProduct/GetProduct.csproj index ba213d4..441cb1e 100755 --- a/src/NET6TopLevelStatements/GetProduct/GetProduct.csproj +++ b/src/NET6TopLevelStatements/GetProduct/GetProduct.csproj @@ -7,14 +7,15 @@ true - - - - - - - - + + + + + + + + + diff --git a/src/NET6TopLevelStatements/GetProducts/GetProducts.csproj b/src/NET6TopLevelStatements/GetProducts/GetProducts.csproj index ba213d4..8b25d8b 100755 --- a/src/NET6TopLevelStatements/GetProducts/GetProducts.csproj +++ b/src/NET6TopLevelStatements/GetProducts/GetProducts.csproj @@ -7,14 +7,14 @@ true - - - - - - - - + + + + + + + + diff --git a/src/NET6TopLevelStatements/PutProduct/PutProduct.csproj b/src/NET6TopLevelStatements/PutProduct/PutProduct.csproj index ba213d4..8b25d8b 100755 --- a/src/NET6TopLevelStatements/PutProduct/PutProduct.csproj +++ b/src/NET6TopLevelStatements/PutProduct/PutProduct.csproj @@ -7,14 +7,14 @@ true - - - - - - - - + + + + + + + + diff --git a/src/NET6TopLevelStatements/Shared/Shared.csproj b/src/NET6TopLevelStatements/Shared/Shared.csproj index 9fce83c..fcdc7e6 100755 --- a/src/NET6TopLevelStatements/Shared/Shared.csproj +++ b/src/NET6TopLevelStatements/Shared/Shared.csproj @@ -6,10 +6,10 @@ enable - - - - - + + + + + diff --git a/src/NET6TopLevelStatements/deploy.sh b/src/NET6TopLevelStatements/deploy.sh new file mode 100755 index 0000000..56d3457 --- /dev/null +++ b/src/NET6TopLevelStatements/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet6-toplevel +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET6TopLevelStatements/run-loadtest.sh b/src/NET6TopLevelStatements/run-loadtest.sh new file mode 100755 index 0000000..d6f645b --- /dev/null +++ b/src/NET6TopLevelStatements/run-loadtest.sh @@ -0,0 +1,182 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + +STACK_NAME=dotnet6-toplevel +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA_GETPRODUCTS Name=FunctionName,Value=$LAMBDA_GETPRODUCT Name=FunctionName,Value=$LAMBDA_DELETEPRODUCT Name=FunctionName,Value=$LAMBDA_PUTPRODUCT \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + + cat ./Report/load-test-errors-$1.json + + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA_GETPRODUCTS" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct +RunLoadTest arm64 ApiUrlArm64 LambdaArm64NameGetProducts LambdaArm64NameGetProduct LambdaArm64NameDeleteProduct LambdaArm64NamePutProduct \ No newline at end of file diff --git a/src/NET6TopLevelStatements/template.yaml b/src/NET6TopLevelStatements/template.yaml index c2888aa..2d428cf 100755 --- a/src/NET6TopLevelStatements/template.yaml +++ b/src/NET6TopLevelStatements/template.yaml @@ -1,10 +1,16 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net6-X86-toplevel + arm64FunctionNamePrefix: + Type: String + Default: Net6-Arm64-toplevel Globals: Function: MemorySize: 1024 - Architectures: [!Ref LambdaArchitecture] Runtime: dotnet6 Timeout: 30 Tracing: Active @@ -12,43 +18,38 @@ Globals: Variables: PRODUCT_TABLE_NAME: !Ref Table -Parameters: - LambdaArchitecture: - Type: String - AllowedValues: - - arm64 - - x86_64 - Description: Enter arm64 or x86_64 - Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] CodeUri: ./GetProducts/ Handler: GetProducts Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:Scan - Resource: !GetAtt Table.Arn + - DynamoDBReadPolicy: + TableName: + !Ref Table - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] CodeUri: ./GetProduct/ Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -57,16 +58,18 @@ Resources: Action: dynamodb:GetItem Resource: !GetAtt Table.Arn - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] CodeUri: ./DeleteProduct/ Handler: DeleteProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -77,16 +80,18 @@ Resources: - dynamodb:GetItem Resource: !GetAtt Table.Arn - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] CodeUri: ./PutProduct/ Handler: PutProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -94,36 +99,86 @@ Resources: - Effect: Allow Action: dynamodb:PutItem Resource: !GetAtt Table.Arn +#ARM64 + GetProductsFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + Architectures: [arm64] + CodeUri: ./GetProducts/ + Handler: GetProducts + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64 + Method: GET + Policies: + - DynamoDBReadPolicy: + TableName: + !Ref Table - GenerateLoadTestResults: + GetProductFunctionArm64: Type: AWS::Serverless::Function Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + Architectures: [arm64] + CodeUri: ./GetProduct/ + Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /test-results + Path: /arm64/{id} Method: GET - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-6-top-level-" - LOAD_TEST_TYPE: "NET 6 Top Level Statements" - LAMBDA_ARCHITECTURE: !Ref LambdaArchitecture Policies: - Version: "2012-10-17" Statement: - - Sid: AllowStartQueries - Effect: Allow + - Effect: Allow + Action: dynamodb:GetItem + Resource: !GetAtt Table.Arn + + DeleteProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + Architectures: [arm64] + CodeUri: ./DeleteProduct/ + Handler: DeleteProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: DELETE + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + - dynamodb:DeleteItem + - dynamodb:GetItem + Resource: !GetAtt Table.Arn + + PutProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] + Architectures: [arm64] + CodeUri: ./PutProduct/ + Handler: PutProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: PUT + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: dynamodb:PutItem + Resource: !GetAtt Table.Arn Table: Type: AWS::DynamoDB::Table @@ -135,10 +190,42 @@ Resources: KeySchema: - AttributeName: id KeyType: HASH - StreamSpecification: - StreamViewType: NEW_AND_OLD_IMAGES Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + + #arm64 + LambdaArm64NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + LambdaArm64NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + LambdaArm64NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + LambdaArm64NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET6WithPowerTools/DeleteProduct/DeleteProduct.csproj b/src/NET6WithPowerTools/DeleteProduct/DeleteProduct.csproj index 9e91e7c..d81c654 100755 --- a/src/NET6WithPowerTools/DeleteProduct/DeleteProduct.csproj +++ b/src/NET6WithPowerTools/DeleteProduct/DeleteProduct.csproj @@ -7,11 +7,11 @@ - - - - - + + + + + diff --git a/src/NET6WithPowerTools/GenerateLoadTestResults/DateUtils.cs b/src/NET6WithPowerTools/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET6WithPowerTools/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET6WithPowerTools/GenerateLoadTestResults/Function.cs b/src/NET6WithPowerTools/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET6WithPowerTools/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET6WithPowerTools/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET6WithPowerTools/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET6WithPowerTools/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET6WithPowerTools/GenerateLoadTestResults/QueryResult.cs b/src/NET6WithPowerTools/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET6WithPowerTools/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET6WithPowerTools/GetProduct/GetProduct.csproj b/src/NET6WithPowerTools/GetProduct/GetProduct.csproj index 9e91e7c..d81c654 100755 --- a/src/NET6WithPowerTools/GetProduct/GetProduct.csproj +++ b/src/NET6WithPowerTools/GetProduct/GetProduct.csproj @@ -7,11 +7,11 @@ - - - - - + + + + + diff --git a/src/NET6WithPowerTools/GetProducts/GetProducts.csproj b/src/NET6WithPowerTools/GetProducts/GetProducts.csproj index 9e91e7c..d81c654 100755 --- a/src/NET6WithPowerTools/GetProducts/GetProducts.csproj +++ b/src/NET6WithPowerTools/GetProducts/GetProducts.csproj @@ -7,11 +7,11 @@ - - - - - + + + + + diff --git a/src/NET6WithPowerTools/PutProduct/PutProduct.csproj b/src/NET6WithPowerTools/PutProduct/PutProduct.csproj index 9e91e7c..d81c654 100755 --- a/src/NET6WithPowerTools/PutProduct/PutProduct.csproj +++ b/src/NET6WithPowerTools/PutProduct/PutProduct.csproj @@ -7,11 +7,11 @@ - - - - - + + + + + diff --git a/src/NET6WithPowerTools/Shared/Shared.csproj b/src/NET6WithPowerTools/Shared/Shared.csproj index e1fbf59..e3a7858 100755 --- a/src/NET6WithPowerTools/Shared/Shared.csproj +++ b/src/NET6WithPowerTools/Shared/Shared.csproj @@ -4,8 +4,8 @@ netstandard2.1 - - - + + + diff --git a/src/NET6WithPowerTools/deploy.sh b/src/NET6WithPowerTools/deploy.sh new file mode 100755 index 0000000..c0d14b6 --- /dev/null +++ b/src/NET6WithPowerTools/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet6-powertools +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET6WithPowerTools/run-loadtest.sh b/src/NET6WithPowerTools/run-loadtest.sh new file mode 100755 index 0000000..0e7e6b9 --- /dev/null +++ b/src/NET6WithPowerTools/run-loadtest.sh @@ -0,0 +1,182 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to yes cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat +#$4 - ARN of sns topic to notify test results + +STACK_NAME=dotnet6-powertools +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +if [ "x$4" != x ]; +then + SNS_TOPIC_ARN=$4 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo SNS_TOPIC_ARN: $SNS_TOPIC_ARN +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + echo -------------------------------------------- + echo GET ERROR METRICS $1 + echo -------------------------------------------- + aws cloudwatch get-metric-statistics \ + --namespace AWS/Lambda \ + --metric-name Errors \ + --dimensions Name=FunctionName,Value=$LAMBDA_GETPRODUCTS Name=FunctionName,Value=$LAMBDA_GETPRODUCT Name=FunctionName,Value=$LAMBDA_DELETEPRODUCT Name=FunctionName,Value=$LAMBDA_PUTPRODUCT \ + --statistics Sum --period 43200 \ + --start-time $startdate --end-time $enddate > ./Report/load-test-errors-$1.json + + cat ./Report/load-test-errors-$1.json + + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json + + if [ "x$SNS_TOPIC_ARN" != x ]; + then + echo -------------------------------------------- + echo Sending message to sns topic: $SNS_TOPIC_ARN + echo -------------------------------------------- + msg=$(<./Report/load-test-report-$1.txt)\n\n$(<./Report/load-test-errors-$1.json) + subject="serverless dotnet demo load test result for $LAMBDA_GETPRODUCTS" + aws sns publish --topic-arn $SNS_TOPIC_ARN --subject "$subject" --message "$msg" + fi +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct +RunLoadTest arm64 ApiUrlArm64 LambdaArm64NameGetProducts LambdaArm64NameGetProduct LambdaArm64NameDeleteProduct LambdaArm64NamePutProduct \ No newline at end of file diff --git a/src/NET6WithPowerTools/template.yaml b/src/NET6WithPowerTools/template.yaml index 0b46b51..00902b6 100755 --- a/src/NET6WithPowerTools/template.yaml +++ b/src/NET6WithPowerTools/template.yaml @@ -1,10 +1,16 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net6-X86-powertools + arm64FunctionNamePrefix: + Type: String + Default: Net6-Arm64-powertools Globals: Function: MemorySize: 1024 - Architectures: [!Ref LambdaArchitecture] Runtime: dotnet6 Timeout: 30 Tracing: Active @@ -17,43 +23,38 @@ Globals: POWERTOOLS_LOGGER_LOG_EVENT: true POWERTOOLS_LOGGER_CASE: SnakeCase -Parameters: - LambdaArchitecture: - Type: String - AllowedValues: - - arm64 - - x86_64 - Description: Enter arm64 or x86_64 - Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] CodeUri: ./GetProducts/ Handler: GetProducts::GetProducts.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - - Version: "2012-10-17" - Statement: - - Effect: Allow - Action: dynamodb:Scan - Resource: !GetAtt Table.Arn + - DynamoDBReadPolicy: + TableName: + !Ref Table - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] CodeUri: ./GetProduct/ Handler: GetProduct::GetProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -62,16 +63,18 @@ Resources: Action: dynamodb:GetItem Resource: !GetAtt Table.Arn - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] CodeUri: ./DeleteProduct/ Handler: DeleteProduct::DeleteProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -82,16 +85,18 @@ Resources: - dynamodb:GetItem Resource: !GetAtt Table.Arn - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] CodeUri: ./PutProduct/ Handler: PutProduct::PutProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -99,36 +104,86 @@ Resources: - Effect: Allow Action: dynamodb:PutItem Resource: !GetAtt Table.Arn +#ARM64 + GetProductsFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + Architectures: [arm64] + CodeUri: ./GetProducts/ + Handler: GetProducts::GetProducts.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64 + Method: GET + Policies: + - DynamoDBReadPolicy: + TableName: + !Ref Table - GenerateLoadTestResults: + GetProductFunctionArm64: Type: AWS::Serverless::Function Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + Architectures: [arm64] + CodeUri: ./GetProduct/ + Handler: GetProduct::GetProduct.Function::FunctionHandler Events: Api: Type: HttpApi Properties: - Path: /test-results + Path: /arm64/{id} Method: GET - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-6-power-tools-" - LOAD_TEST_TYPE: "NET 6 w/Power Tools" - LAMBDA_ARCHITECTURE: !Ref LambdaArchitecture Policies: - Version: "2012-10-17" Statement: - - Sid: AllowStartQueries - Effect: Allow + - Effect: Allow + Action: dynamodb:GetItem + Resource: !GetAtt Table.Arn + + DeleteProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + Architectures: [arm64] + CodeUri: ./DeleteProduct/ + Handler: DeleteProduct::DeleteProduct.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: DELETE + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + - dynamodb:DeleteItem + - dynamodb:GetItem + Resource: !GetAtt Table.Arn + + PutProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] + Architectures: [arm64] + CodeUri: ./PutProduct/ + Handler: PutProduct::PutProduct.Function::FunctionHandler + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: PUT + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: dynamodb:PutItem + Resource: !GetAtt Table.Arn Table: Type: AWS::DynamoDB::Table @@ -140,10 +195,42 @@ Resources: KeySchema: - AttributeName: id KeyType: HASH - StreamSpecification: - StreamViewType: NEW_AND_OLD_IMAGES Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + + #arm64 + LambdaArm64NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + LambdaArm64NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + LambdaArm64NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + LambdaArm64NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET8/DeleteProduct/DeleteProduct.csproj b/src/NET8/DeleteProduct/DeleteProduct.csproj index 825e086..a386b03 100644 --- a/src/NET8/DeleteProduct/DeleteProduct.csproj +++ b/src/NET8/DeleteProduct/DeleteProduct.csproj @@ -15,12 +15,12 @@ - - - - - - + + + + + + diff --git a/src/NET8/DeleteProduct/aws-lambda-tools-defaults.json b/src/NET8/DeleteProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8/DeleteProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8/GenerateLoadTestResults/DateUtils.cs b/src/NET8/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET8/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET8/GenerateLoadTestResults/Function.cs b/src/NET8/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET8/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET8/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET8/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET8/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET8/GenerateLoadTestResults/QueryResult.cs b/src/NET8/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET8/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET8/GetProduct/GetProduct.csproj b/src/NET8/GetProduct/GetProduct.csproj index 9757d2d..8253d9e 100644 --- a/src/NET8/GetProduct/GetProduct.csproj +++ b/src/NET8/GetProduct/GetProduct.csproj @@ -13,12 +13,12 @@ linux-x64 - - - - - - + + + + + + diff --git a/src/NET8/GetProduct/aws-lambda-tools-defaults.json b/src/NET8/GetProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8/GetProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8/GetProducts/GetProducts.csproj b/src/NET8/GetProducts/GetProducts.csproj index 55460a1..982b1ac 100644 --- a/src/NET8/GetProducts/GetProducts.csproj +++ b/src/NET8/GetProducts/GetProducts.csproj @@ -14,12 +14,12 @@ linux-x64 - - - - - - + + + + + + diff --git a/src/NET8/GetProducts/aws-lambda-tools-defaults.json b/src/NET8/GetProducts/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8/GetProducts/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8/Makefile b/src/NET8/Makefile index 2aadd31..4850513 100644 --- a/src/NET8/Makefile +++ b/src/NET8/Makefile @@ -1,15 +1,23 @@ -build-GetProductFunction: - dotnet clean +build-GetProductFunctionX86: dotnet publish GetProduct/GetProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-GetProductsFunction: - dotnet clean +build-GetProductsFunctionX86: dotnet publish GetProducts/GetProducts.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-PutProductFunction: - dotnet clean +build-PutProductFunctionX86: dotnet publish PutProduct/PutProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-DeleteProductFunction: - dotnet clean - dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file +build-DeleteProductFunctionX86: + dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) + +build-GetProductFunctionArm64: + dotnet publish GetProduct/GetProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-GetProductsFunctionArm64: + dotnet publish GetProducts/GetProducts.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-PutProductFunctionArm64: + dotnet publish PutProduct/PutProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) + +build-DeleteProductFunctionArm64: + dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file diff --git a/src/NET8/PutProduct/PutProduct.csproj b/src/NET8/PutProduct/PutProduct.csproj index 9fae9e3..3f79dae 100644 --- a/src/NET8/PutProduct/PutProduct.csproj +++ b/src/NET8/PutProduct/PutProduct.csproj @@ -15,12 +15,12 @@ - - - - - - + + + + + + diff --git a/src/NET8/PutProduct/aws-lambda-tools-defaults.json b/src/NET8/PutProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 7ed1688..0000000 --- a/src/NET8/PutProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "Information": [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - "dotnet lambda help", - "All the command line options for the Lambda command can be specified in this file." - ], - "profile": "", - "region": "", - "configuration": "Release", - "function-runtime": "provided.al2", - "function-memory-size": 256, - "function-timeout": 30, - "function-handler": "bootstrap" -} \ No newline at end of file diff --git a/src/NET8/Shared/Shared.csproj b/src/NET8/Shared/Shared.csproj index 31e5c9f..2797db9 100644 --- a/src/NET8/Shared/Shared.csproj +++ b/src/NET8/Shared/Shared.csproj @@ -13,10 +13,10 @@ linux-x64 - - - - + + + + diff --git a/src/NET8/deploy.sh b/src/NET8/deploy.sh new file mode 100755 index 0000000..814ecb5 --- /dev/null +++ b/src/NET8/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet8 +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET8/run-loadtest.sh b/src/NET8/run-loadtest.sh new file mode 100755 index 0000000..2391e84 --- /dev/null +++ b/src/NET8/run-loadtest.sh @@ -0,0 +1,153 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat + +STACK_NAME=dotnet8 +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct +RunLoadTest arm64 ApiUrlArm64 LambdaArm64NameGetProducts LambdaArm64NameGetProduct LambdaArm64NameDeleteProduct LambdaArm64NamePutProduct \ No newline at end of file diff --git a/src/NET8/template.yaml b/src/NET8/template.yaml index c6c34f7..0e49fe0 100644 --- a/src/NET8/template.yaml +++ b/src/NET8/template.yaml @@ -1,10 +1,17 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net8-X86 + arm64FunctionNamePrefix: + Type: String + Default: Net8-Arm64 + Globals: Function: MemorySize: 1024 - Architectures: ["x86_64"] Runtime: provided.al2 Timeout: 30 Tracing: Active @@ -13,16 +20,19 @@ Globals: PRODUCT_TABLE_NAME: !Ref Table Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] CodeUri: ./ Handler: bootstrap Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - DynamoDBReadPolicy: @@ -31,16 +41,18 @@ Resources: Metadata: BuildMethod: makefile - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -51,16 +63,18 @@ Resources: Metadata: BuildMethod: makefile - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: DeleteProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -73,16 +87,18 @@ Resources: Metadata: BuildMethod: makefile - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] CodeUri: ./ Handler: PutProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -92,37 +108,94 @@ Resources: Resource: !GetAtt Table.Arn Metadata: BuildMethod: makefile + #ARM64 + GetProductsFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + Architectures: [arm64] + CodeUri: ./ + Handler: bootstrap + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64 + Method: GET + Policies: + - DynamoDBReadPolicy: + TableName: + !Ref Table + Metadata: + BuildMethod: makefile - GenerateLoadTestResults: + GetProductFunctionArm64: Type: AWS::Serverless::Function Properties: - CodeUri: ./GenerateLoadTestResults/ - Handler: GenerateLoadTestResults::GenerateLoadTestResults.Function::FunctionHandler - Runtime: dotnet6 + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /test-results + Path: /arm64/{id} Method: GET - Environment: - Variables: - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-8-base-" - LOAD_TEST_TYPE: "NET 8" - LAMBDA_ARCHITECTURE: "x86_64" Policies: - Version: "2012-10-17" Statement: - - Sid: AllowStartQueries - Effect: Allow + - Effect: Allow + Action: dynamodb:GetItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile + + DeleteProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: DeleteProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: DELETE + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + - dynamodb:DeleteItem + - dynamodb:GetItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile + + PutProductFunctionArm64: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] + Architectures: [arm64] + CodeUri: ./ + Handler: PutProduct + Events: + Api: + Type: HttpApi + Properties: + Path: /arm64/{id} + Method: PUT + Policies: + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: dynamodb:PutItem + Resource: !GetAtt Table.Arn + Metadata: + BuildMethod: makefile Table: Type: AWS::DynamoDB::Table @@ -138,4 +211,38 @@ Resources: Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + + #arm64 + LambdaArm64NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProducts]] + LambdaArm64NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,GetProduct]] + LambdaArm64NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,DeleteProduct]] + LambdaArm64NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref arm64FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET8MinimalAPI/ApiBootstrap/ApiBootstrap.csproj b/src/NET8MinimalAPI/ApiBootstrap/ApiBootstrap.csproj index cf7e8a5..c3879a4 100644 --- a/src/NET8MinimalAPI/ApiBootstrap/ApiBootstrap.csproj +++ b/src/NET8MinimalAPI/ApiBootstrap/ApiBootstrap.csproj @@ -5,7 +5,7 @@ net8.0 true Lambda - bootstrap + bootstrap true false true @@ -15,14 +15,14 @@ - - - - - - - - + + + + + + + + diff --git a/src/NET8MinimalAPI/ApiBootstrap/CloudWatchQueryExecution.cs b/src/NET8MinimalAPI/ApiBootstrap/CloudWatchQueryExecution.cs deleted file mode 100644 index 23b847f..0000000 --- a/src/NET8MinimalAPI/ApiBootstrap/CloudWatchQueryExecution.cs +++ /dev/null @@ -1,51 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.Core; - -namespace GetProducts; - -public static class CloudWatchQueryExecution -{ - public static async Task>> RunQuery(AmazonCloudWatchLogsClient cloudWatchLogsClient) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - var logGroupList = await cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - var queryRes = await cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - var queryResults = await cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - return finalResults; - } -} \ No newline at end of file diff --git a/src/NET8MinimalAPI/ApiBootstrap/DateUtils.cs b/src/NET8MinimalAPI/ApiBootstrap/DateUtils.cs deleted file mode 100644 index 8a572e0..0000000 --- a/src/NET8MinimalAPI/ApiBootstrap/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GetProducts; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET8MinimalAPI/ApiBootstrap/Function.cs b/src/NET8MinimalAPI/ApiBootstrap/Function.cs index a4cd497..10a82a7 100644 --- a/src/NET8MinimalAPI/ApiBootstrap/Function.cs +++ b/src/NET8MinimalAPI/ApiBootstrap/Function.cs @@ -4,7 +4,6 @@ using System.Text.Json; using Amazon.CloudWatchLogs; using Amazon.CloudWatchLogs.Model; -using GetProducts; using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.DependencyInjection; @@ -14,6 +13,15 @@ using Shared.Models; var app = Startup.Build(args); +var basePath=Environment.GetEnvironmentVariable("DEMO_BASE_PATH"); +if(String.IsNullOrEmpty(basePath)) + app.Logger.LogInformation($"No BASE PATH specified"); +else +{ + app.Logger.LogInformation($"Using BASE PATH:{basePath}"); + app.UsePathBase(basePath); + app.UseRouting(); +} var dataAccess = app.Services.GetRequiredService(); @@ -37,7 +45,7 @@ { var id = context.Request.RouteValues["id"].ToString(); - app.Logger.LogInformation($"Received request to delete {id}"); + app.Logger.LogInformation($"Received request to delete {id} from the database"); var product = await dataAccess.GetProduct(id); @@ -123,45 +131,4 @@ await context.Response.WriteAsJsonAsync(product); }); -app.MapGet("/test-results", async (HttpContext context) => -{ - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await CloudWatchQueryExecution.RunQuery(cloudWatchClient); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - context.Response.StatusCode = (int) HttpStatusCode.OK; - await context.Response.WriteAsync(wrapper.AsMarkdownTableRow()); -}); - app.Run(); \ No newline at end of file diff --git a/src/NET8MinimalAPI/ApiBootstrap/QueryResult.cs b/src/NET8MinimalAPI/ApiBootstrap/QueryResult.cs deleted file mode 100644 index ff692d2..0000000 --- a/src/NET8MinimalAPI/ApiBootstrap/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GetProducts; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET8MinimalAPI/Makefile b/src/NET8MinimalAPI/Makefile index 3f5ee87..ae88de0 100644 --- a/src/NET8MinimalAPI/Makefile +++ b/src/NET8MinimalAPI/Makefile @@ -1,3 +1,4 @@ -build-ApiFunction: - dotnet clean +build-MinimalApiX86: dotnet publish ApiBootstrap/ApiBootstrap.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) +build-MinimalApiArm64: + dotnet publish ApiBootstrap/ApiBootstrap.csproj -c Release -r linux-arm64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file diff --git a/src/NET8MinimalAPI/Shared/Shared.csproj b/src/NET8MinimalAPI/Shared/Shared.csproj index 34cd820..7d6a8a2 100644 --- a/src/NET8MinimalAPI/Shared/Shared.csproj +++ b/src/NET8MinimalAPI/Shared/Shared.csproj @@ -1,14 +1,14 @@ - net8.0 + net8.0 enable enable - - + + diff --git a/src/NET8MinimalAPI/deploy.sh b/src/NET8MinimalAPI/deploy.sh new file mode 100755 index 0000000..1caf08f --- /dev/null +++ b/src/NET8MinimalAPI/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet8-minimal-api +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build +sam deploy --stack-name dotnet8-minimal-api --resolve-s3 --s3-prefix dotnet8-minimal-api --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET8MinimalAPI/omnisharp.json b/src/NET8MinimalAPI/omnisharp.json deleted file mode 100644 index c42f8db..0000000 --- a/src/NET8MinimalAPI/omnisharp.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "fileOptions": { - "excludeSearchPatterns": [ - "**/bin/**/*", - "**/obj/**/*" - ] - }, - "msbuild": { - "Platform": "rhel.7.2-x64" - } -} \ No newline at end of file diff --git a/src/NET8MinimalAPI/run-loadtest.sh b/src/NET8MinimalAPI/run-loadtest.sh new file mode 100755 index 0000000..4c69913 --- /dev/null +++ b/src/NET8MinimalAPI/run-loadtest.sh @@ -0,0 +1,123 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat + +STACK_NAME=dotnet8-minimal-api +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $LAMBDA: $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + QUERY_ID=$(aws logs start-query \ + --log-group-name /aws/lambda/$LAMBDA \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $LAMBDA + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS lambda: $LAMBDA >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json +} + +RunLoadTest x86 ApiUrlX86 LambdaX86Name +RunLoadTest arm64 ApiUrlArm64 LambdaArm64Name \ No newline at end of file diff --git a/src/NET8MinimalAPI/template.yaml b/src/NET8MinimalAPI/template.yaml index aeb8b06..abb40f5 100644 --- a/src/NET8MinimalAPI/template.yaml +++ b/src/NET8MinimalAPI/template.yaml @@ -1,47 +1,69 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionName: + Type: String + Default: Net8-MinimalApi-X86 + arm64FunctionName: + Type: String + Default: Net8-MinimalApi-Arm64 + Globals: Function: MemorySize: 1024 - Architectures: ["x86_64"] Runtime: provided.al2 Timeout: 30 Tracing: Active Environment: Variables: PRODUCT_TABLE_NAME: !Ref Table - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-8-minimal-" - LOAD_TEST_TYPE: "NET 8 Minimal API" Resources: - ApiFunction: + MinimalApiX86: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Ref x86FunctionName + Architectures: [x86_64] + CodeUri: ./ + Handler: ApiBootstrap + Environment: + Variables: + DEMO_BASE_PATH: /x86 + Events: + Api: + Type: HttpApi + Properties: + Path: /x86/{proxy+} + Method: ANY + Policies: + - DynamoDBCrudPolicy: + TableName: + !Ref Table + Metadata: + BuildMethod: makefile + MinimalApiArm64: Type: AWS::Serverless::Function Properties: + FunctionName: !Ref arm64FunctionName + Architectures: [arm64] CodeUri: ./ Handler: ApiBootstrap + Environment: + Variables: + DEMO_BASE_PATH: /arm64 Events: Api: Type: HttpApi Properties: - Path: /{proxy+} + Path: /arm64/{proxy+} Method: ANY Policies: - DynamoDBCrudPolicy: TableName: !Ref Table - - Version: "2012-10-17" - Statement: - - Sid: AllowStartQueries - Effect: Allow - Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" + Metadata: + BuildMethod: makefile Table: Type: AWS::DynamoDB::Table @@ -59,4 +81,16 @@ Resources: Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + LambdaX86Name: + Description: "Lambda X86 Name" + Value: !Ref x86FunctionName + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + LambdaArm64Name: + Description: "Lambda Arm64 Name" + Value: !Ref arm64FunctionName + ApiUrlArm64: + Description: "Arm64 GateAPI endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/arm64" \ No newline at end of file diff --git a/src/NET8Native/DeleteProduct/DeleteProduct.csproj b/src/NET8Native/DeleteProduct/DeleteProduct.csproj index 1be8350..9f5a81b 100644 --- a/src/NET8Native/DeleteProduct/DeleteProduct.csproj +++ b/src/NET8Native/DeleteProduct/DeleteProduct.csproj @@ -3,25 +3,29 @@ exe net8.0 - true - Lambda bootstrap + true - false - true - true + true + Lambda + true + skylake Speed + + false + true + true - - - - - - + + + + + + diff --git a/src/NET8Native/DeleteProduct/aws-lambda-tools-defaults.json b/src/NET8Native/DeleteProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8Native/DeleteProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8Native/GenerateLoadTestResults/DateUtils.cs b/src/NET8Native/GenerateLoadTestResults/DateUtils.cs deleted file mode 100644 index c37c43d..0000000 --- a/src/NET8Native/GenerateLoadTestResults/DateUtils.cs +++ /dev/null @@ -1,13 +0,0 @@ -using System; - -namespace GenerateLoadTestResults; - -public static class DateUtils -{ - public static long AsUnixTimestamp(this DateTime date) - { - DateTime origin = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc); - TimeSpan diff = date.ToUniversalTime() - origin; - return (long)Math.Floor(diff.TotalSeconds); - } -} \ No newline at end of file diff --git a/src/NET8Native/GenerateLoadTestResults/Function.cs b/src/NET8Native/GenerateLoadTestResults/Function.cs deleted file mode 100644 index 21a57a8..0000000 --- a/src/NET8Native/GenerateLoadTestResults/Function.cs +++ /dev/null @@ -1,146 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Net; -using System.Net.Http; -using System.Text.Json; -using System.Threading.Tasks; -using Amazon.CloudWatchLogs; -using Amazon.CloudWatchLogs.Model; -using Amazon.Lambda.APIGatewayEvents; -using Amazon.Lambda.Core; - -[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] - -namespace GenerateLoadTestResults -{ - public class Function - { - private AmazonCloudWatchLogsClient _cloudWatchLogsClient; - - public Function() - { - this._cloudWatchLogsClient = new AmazonCloudWatchLogsClient(); - } - - public async Task FunctionHandler( - APIGatewayHttpApiV2ProxyRequest apigProxyEvent, - ILambdaContext context) - { - if (!apigProxyEvent.RequestContext.Http.Method.Equals(HttpMethod.Get.Method)) - { - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Only GET allowed", - StatusCode = (int) HttpStatusCode.MethodNotAllowed, - }; - } - - try - { - var resultRows = 0; - var queryCount = 0; - - List> finalResults = new List>(); - - while (resultRows < 2 || queryCount >= 3) - { - finalResults = await runQuery(context); - - resultRows = finalResults.Count; - queryCount++; - } - - var wrapper = new QueryResultWrapper() - { - LoadTestType = - $"{Environment.GetEnvironmentVariable("LOAD_TEST_TYPE")} ({Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")})", - WarmStart = new QueryResult() - { - Count = finalResults[0][1].Value, - P50 = finalResults[0][2].Value, - P90 = finalResults[0][3].Value, - P99 = finalResults[0][4].Value, - Max = finalResults[0][5].Value, - }, - ColdStart = new QueryResult() - { - Count = finalResults[1][1].Value, - P50 = finalResults[1][2].Value, - P90 = finalResults[1][3].Value, - P99 = finalResults[1][4].Value, - Max = finalResults[1][5].Value, - } - }; - - return new APIGatewayHttpApiV2ProxyResponse - { - StatusCode = (int) HttpStatusCode.OK, - Body = wrapper.AsMarkdownTableRow(), - Headers = new Dictionary {{"Content-Type", "text/html"}} - }; - } - catch (Exception e) - { - context.Logger.LogLine($"Error retrieving results {e.Message} {e.StackTrace}"); - - return new APIGatewayHttpApiV2ProxyResponse - { - Body = "Not Found", - StatusCode = (int) HttpStatusCode.InternalServerError, - }; - } - } - - private async Task>> runQuery(ILambdaContext context) - { - var logGroupNamePrefix = - $"{Environment.GetEnvironmentVariable("LOG_GROUP_PREFIX")}{Environment.GetEnvironmentVariable("LAMBDA_ARCHITECTURE")}" - .Replace("_", "-"); - - context.Logger.LogLine($"Retrieving log groups with prefix {logGroupNamePrefix}"); - - var logGroupList = await _cloudWatchLogsClient.DescribeLogGroupsAsync(new DescribeLogGroupsRequest() - { - LogGroupNamePrefix = logGroupNamePrefix, - }); - - context.Logger.LogLine($"Found {logGroupList.LogGroups.Count} log group(s)"); - - var queryRes = await _cloudWatchLogsClient.StartQueryAsync(new StartQueryRequest() - { - LogGroupNames = logGroupList.LogGroups.Select(p => p.LogGroupName).ToList(), - QueryString = - "filter @type=\"REPORT\" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart", - StartTime = DateTime.Now.AddMinutes(-20).AsUnixTimestamp(), - EndTime = DateTime.Now.AsUnixTimestamp(), - }); - - context.Logger.LogLine($"Running query, query id is {queryRes.QueryId}"); - - QueryStatus currentQueryStatus = QueryStatus.Running; - List> finalResults = new List>(); - - while (currentQueryStatus == QueryStatus.Running || currentQueryStatus == QueryStatus.Scheduled) - { - context.Logger.LogLine("Retrieving query results"); - - var queryResults = await _cloudWatchLogsClient.GetQueryResultsAsync(new GetQueryResultsRequest() - { - QueryId = queryRes.QueryId - }); - - context.Logger.LogLine($"Query result status is {queryResults.Status}"); - - currentQueryStatus = queryResults.Status; - finalResults = queryResults.Results; - - await Task.Delay(TimeSpan.FromSeconds(5)); - } - - context.Logger.LogLine($"Final results: {finalResults.Count} row(s)"); - - return finalResults; - } - } -} \ No newline at end of file diff --git a/src/NET8Native/GenerateLoadTestResults/GenerateLoadTestResults.csproj b/src/NET8Native/GenerateLoadTestResults/GenerateLoadTestResults.csproj deleted file mode 100644 index 4a8d708..0000000 --- a/src/NET8Native/GenerateLoadTestResults/GenerateLoadTestResults.csproj +++ /dev/null @@ -1,17 +0,0 @@ - - - - net6.0 - true - true - - - - - - - - - - - diff --git a/src/NET8Native/GenerateLoadTestResults/QueryResult.cs b/src/NET8Native/GenerateLoadTestResults/QueryResult.cs deleted file mode 100644 index 69b5a31..0000000 --- a/src/NET8Native/GenerateLoadTestResults/QueryResult.cs +++ /dev/null @@ -1,25 +0,0 @@ -namespace GenerateLoadTestResults; - -public record QueryResultWrapper -{ - public string LoadTestType { get; set; } - - public QueryResult ColdStart { get; set; } - - public QueryResult WarmStart { get; set; } - - public string AsMarkdownTableRow() => $"
Cold Start (ms)Warm Start (ms)
p50p90p99maxp50p90p99max
{LoadTestType}{ColdStart.P50}{ColdStart.P90}{ColdStart.P99}{ColdStart.Max}{WarmStart.P50}{WarmStart.P90}{WarmStart.P99}{WarmStart.Max}
"; -} - -public record QueryResult -{ - public string Count { get; set; } - - public string P50 { get; set; } - - public string P90 { get; set; } - - public string P99 { get; set; } - - public string Max { get; set; } -} \ No newline at end of file diff --git a/src/NET8Native/GetProduct/GetProduct.csproj b/src/NET8Native/GetProduct/GetProduct.csproj index 1354b30..364daed 100644 --- a/src/NET8Native/GetProduct/GetProduct.csproj +++ b/src/NET8Native/GetProduct/GetProduct.csproj @@ -1,29 +1,29 @@  - exe net8.0 - true - Lambda bootstrap true - false - true - true - + true + Lambda + true + skylake Speed + + false + true + true - - - - - - + + + + + + diff --git a/src/NET8Native/GetProduct/aws-lambda-tools-defaults.json b/src/NET8Native/GetProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8Native/GetProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8Native/GetProducts/GetProducts.csproj b/src/NET8Native/GetProducts/GetProducts.csproj index 578c023..364daed 100644 --- a/src/NET8Native/GetProducts/GetProducts.csproj +++ b/src/NET8Native/GetProducts/GetProducts.csproj @@ -1,26 +1,29 @@  - exe net8.0 - true - Lambda bootstrap + true - false - true - true + true + Lambda + true + skylake Speed + + false + true + true - - - - - - + + + + + + diff --git a/src/NET8Native/GetProducts/aws-lambda-tools-defaults.json b/src/NET8Native/GetProducts/aws-lambda-tools-defaults.json deleted file mode 100644 index 1d112e0..0000000 --- a/src/NET8Native/GetProducts/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "Information" : [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - - "dotnet lambda help", - - "All the command line options for the Lambda command can be specified in this file." - ], - - "profile":"", - "region" : "", - "configuration": "Release", - "function-runtime":"provided.al2", - "function-memory-size" : 256, - "function-timeout" : 30, - "function-handler" : "CancelBooking" -} diff --git a/src/NET8Native/Makefile b/src/NET8Native/Makefile index 9cc4200..9993dc3 100644 --- a/src/NET8Native/Makefile +++ b/src/NET8Native/Makefile @@ -1,11 +1,11 @@ -build-GetProductsFunction: - dotnet publish -c Release -r linux-x64 ./GetProducts/GetProducts.csproj -o $(ARTIFACTS_DIR) +build-GetProductFunctionX86: + dotnet publish GetProduct/GetProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-GetProductFunction: - dotnet publish -c Release -r linux-x64 ./GetProduct/GetProduct.csproj -o $(ARTIFACTS_DIR) +build-GetProductsFunctionX86: + dotnet publish GetProducts/GetProducts.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-DeleteProductFunction: - dotnet publish -c Release -r linux-x64 ./DeleteProduct/DeleteProduct.csproj -o $(ARTIFACTS_DIR) +build-PutProductFunctionX86: + dotnet publish PutProduct/PutProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) -build-PutProductFunction: - dotnet publish -c Release -r linux-x64 ./PutProduct/PutProduct.csproj -o $(ARTIFACTS_DIR) \ No newline at end of file +build-DeleteProductFunctionX86: + dotnet publish DeleteProduct/DeleteProduct.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file diff --git a/src/NET8Native/PutProduct/PutProduct.csproj b/src/NET8Native/PutProduct/PutProduct.csproj index 1be8350..9f5a81b 100644 --- a/src/NET8Native/PutProduct/PutProduct.csproj +++ b/src/NET8Native/PutProduct/PutProduct.csproj @@ -3,25 +3,29 @@ exe net8.0 - true - Lambda bootstrap + true - false - true - true + true + Lambda + true + skylake Speed + + false + true + true - - - - - - + + + + + + diff --git a/src/NET8Native/PutProduct/aws-lambda-tools-defaults.json b/src/NET8Native/PutProduct/aws-lambda-tools-defaults.json deleted file mode 100644 index 7ed1688..0000000 --- a/src/NET8Native/PutProduct/aws-lambda-tools-defaults.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "Information": [ - "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.", - "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.", - "dotnet lambda help", - "All the command line options for the Lambda command can be specified in this file." - ], - "profile": "", - "region": "", - "configuration": "Release", - "function-runtime": "provided.al2", - "function-memory-size": 256, - "function-timeout": 30, - "function-handler": "bootstrap" -} \ No newline at end of file diff --git a/src/NET8Native/Shared/JsonSerializerContext.cs b/src/NET8Native/Shared/JsonSerializerContext.cs index 99f168f..f71528f 100644 --- a/src/NET8Native/Shared/JsonSerializerContext.cs +++ b/src/NET8Native/Shared/JsonSerializerContext.cs @@ -1,4 +1,5 @@ -using System.Text.Json.Serialization; +using System.Collections.Generic; +using System.Text.Json.Serialization; using Amazon.Lambda.APIGatewayEvents; using Shared.Models; diff --git a/src/NET8Native/Shared/Shared.csproj b/src/NET8Native/Shared/Shared.csproj index a2b16db..84e7fb3 100644 --- a/src/NET8Native/Shared/Shared.csproj +++ b/src/NET8Native/Shared/Shared.csproj @@ -2,26 +2,25 @@ net8.0 - enable - enable - true - true - false - true - true - + true + Lambda + true + skylake Speed + + false + true + true - - - - + + + + diff --git a/src/NET8Native/deploy.sh b/src/NET8Native/deploy.sh new file mode 100755 index 0000000..a6d0c85 --- /dev/null +++ b/src/NET8Native/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet8-native +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build --use-container --build-image plantpowerjames/dotnet-8-lambda-build:rc2 +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET8Native/run-loadtest.sh b/src/NET8Native/run-loadtest.sh new file mode 100755 index 0000000..dd87955 --- /dev/null +++ b/src/NET8Native/run-loadtest.sh @@ -0,0 +1,152 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat + +STACK_NAME=dotnet8-native +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name GetProducts + #$4 - Stack output name to get lambda name GetProduct + #$5 - Stack output name to get lambda name DeleteProduct + #$6 - Stack output name to get lambda name PutProduct + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA_GETPRODUCTS=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCTS + + LAMBDA_GETPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$4'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_GETPRODUCT + + LAMBDA_DELETEPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$5'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_DELETEPRODUCT + + LAMBDA_PUTPRODUCT=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$6'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA_PUTPRODUCT + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCTS + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_GETPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_DELETEPRODUCT + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA_PUTPRODUCT + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA_PUTPRODUCT + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/codebuild/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + QUERY_ID=$(aws logs start-query \ + --log-group-names "/aws/lambda/$LAMBDA_GETPRODUCTS" "/aws/lambda/$LAMBDA_GETPRODUCT" "/aws/lambda/$LAMBDA_DELETEPRODUCT" "/aws/lambda/$LAMBDA_PUTPRODUCT" \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $1 + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json +} + +RunLoadTest x86 ApiUrlX86 LambdaX86NameGetProducts LambdaX86NameGetProduct LambdaX86NameDeleteProduct LambdaX86NamePutProduct \ No newline at end of file diff --git a/src/NET8Native/template.yaml b/src/NET8Native/template.yaml index 22dc554..88b4d70 100644 --- a/src/NET8Native/template.yaml +++ b/src/NET8Native/template.yaml @@ -1,10 +1,14 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionNamePrefix: + Type: String + Default: Net8-Native-X86 + Globals: Function: MemorySize: 1024 - Architectures: ["x86_64"] Runtime: provided.al2 Timeout: 30 Tracing: Active @@ -13,16 +17,19 @@ Globals: PRODUCT_TABLE_NAME: !Ref Table Resources: - GetProductsFunction: + #X86 + GetProductsFunctionX86: Type: AWS::Serverless::Function Properties: - CodeUri: . + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + Architectures: [x86_64] + CodeUri: ./ Handler: bootstrap Events: Api: Type: HttpApi Properties: - Path: / + Path: /x86 Method: GET Policies: - DynamoDBReadPolicy: @@ -31,16 +38,18 @@ Resources: Metadata: BuildMethod: makefile - GetProductFunction: + GetProductFunctionX86: Type: AWS::Serverless::Function Properties: - CodeUri: . + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + Architectures: [x86_64] + CodeUri: ./ Handler: GetProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: GET Policies: - Version: "2012-10-17" @@ -51,16 +60,18 @@ Resources: Metadata: BuildMethod: makefile - DeleteProductFunction: + DeleteProductFunctionX86: Type: AWS::Serverless::Function Properties: - CodeUri: . + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + Architectures: [x86_64] + CodeUri: ./ Handler: DeleteProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: DELETE Policies: - Version: "2012-10-17" @@ -73,16 +84,18 @@ Resources: Metadata: BuildMethod: makefile - PutProductFunction: + PutProductFunctionX86: Type: AWS::Serverless::Function Properties: - CodeUri: . + FunctionName: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] + Architectures: [x86_64] + CodeUri: ./ Handler: PutProduct Events: Api: Type: HttpApi Properties: - Path: /{id} + Path: /x86/{id} Method: PUT Policies: - Version: "2012-10-17" @@ -107,4 +120,21 @@ Resources: Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/x86" + + #x86 + LambdaX86NameGetProducts: + Description: "Lambda X86 GetProducts" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProducts]] + LambdaX86NameGetProduct: + Description: "Lambda X86 GetProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,GetProduct]] + LambdaX86NameDeleteProduct: + Description: "Lambda X86 DeleteProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,DeleteProduct]] + LambdaX86NamePutProduct: + Description: "Lambda X86 PutProduct" + Value: !Join [ -,[!Ref x86FunctionNamePrefix,PutProduct]] \ No newline at end of file diff --git a/src/NET8NativeMinimalAPI/ApiBootstrap/ApiBootstrap.csproj b/src/NET8NativeMinimalAPI/ApiBootstrap/ApiBootstrap.csproj index 48bb4ef..e2c8e51 100644 --- a/src/NET8NativeMinimalAPI/ApiBootstrap/ApiBootstrap.csproj +++ b/src/NET8NativeMinimalAPI/ApiBootstrap/ApiBootstrap.csproj @@ -3,24 +3,28 @@ Exe net8.0 + bootstrap + + true + + true true enable enable GetProducts - true - bootstrap + full - true - - - - - - - + + + + + + + + @@ -28,6 +32,8 @@ + + diff --git a/src/NET8NativeMinimalAPI/Makefile b/src/NET8NativeMinimalAPI/Makefile index fc76f1e..0e8e9c9 100644 --- a/src/NET8NativeMinimalAPI/Makefile +++ b/src/NET8NativeMinimalAPI/Makefile @@ -1,2 +1,2 @@ -build-ApiFunction: - dotnet publish -c Release -r linux-x64 ./ApiBootstrap/ApiBootstrap.csproj -o $(ARTIFACTS_DIR) \ No newline at end of file +build-MinimalApiX86: + dotnet publish ApiBootstrap/ApiBootstrap.csproj -c Release -r linux-x64 --self-contained -o $(ARTIFACTS_DIR) \ No newline at end of file diff --git a/src/NET8NativeMinimalAPI/Shared/Shared.csproj b/src/NET8NativeMinimalAPI/Shared/Shared.csproj index 8361c85..466b824 100644 --- a/src/NET8NativeMinimalAPI/Shared/Shared.csproj +++ b/src/NET8NativeMinimalAPI/Shared/Shared.csproj @@ -2,13 +2,17 @@ net8.0 + + true + + true enable enable - - + + diff --git a/src/NET8NativeMinimalAPI/deploy.sh b/src/NET8NativeMinimalAPI/deploy.sh new file mode 100755 index 0000000..d6da858 --- /dev/null +++ b/src/NET8NativeMinimalAPI/deploy.sh @@ -0,0 +1,43 @@ +#Arguments: +#$1 - delete stack formation to ensure that all test have same conditon and favour similar ammount of cold start events + +STACK_NAME=dotnet8-native-minimal-api +DELETE_STACK=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + DELETE_STACK=$1 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DELETE_STACK: $DELETE_STACK +echo -------------------------------------------- +echo "${NO_COLOR}" + +if [ $DELETE_STACK == "yes" ]; +then + echo "${COLOR}" + echo -------------------------------------------- + echo DELETING STACK $STACK_NAME + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation delete-stack --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Waiting stack to be deleted + echo -------------------------------------------- + echo "${NO_COLOR}" + aws cloudformation wait stack-delete-complete --stack-name $STACK_NAME + echo "${COLOR}" + echo --------------------------------------------- + echo Stack deleted + echo -------------------------------------------- + echo "${NO_COLOR}" +fi + +sam build --use-container --build-image plantpowerjames/dotnet-8-lambda-build:rc2 +sam deploy --stack-name $STACK_NAME --resolve-s3 --s3-prefix $STACK_NAME --no-confirm-changeset --no-fail-on-empty-changeset --capabilities CAPABILITY_IAM \ No newline at end of file diff --git a/src/NET8NativeMinimalAPI/run-loadtest.sh b/src/NET8NativeMinimalAPI/run-loadtest.sh new file mode 100755 index 0000000..351288a --- /dev/null +++ b/src/NET8NativeMinimalAPI/run-loadtest.sh @@ -0,0 +1,122 @@ +#Arguments: +#$1 - load test duration in seconds +#$2 - log interval to be used in the cloudwatch query in minutes +#$3 - when equal to 1 cloudwatch log group will be deleted to ensure that only logs of the load test will be evaluated for stat + +STACK_NAME=dotnet8-native-minimal-api +TEST_DURATIOMN_SEC=60 +LOG_INTERVAL_MIN=20 +LOG_DELETE=yes + +COLOR='\033[0;33m' +NO_COLOR='\033[0m' # No Color + +if [ "x$1" != x ]; +then + TEST_DURATIOMN_SEC=$1 +fi + +if [ "x$2" != x ]; +then + LOG_INTERVAL_MIN=$2 +fi + +if [ "x$3" != x ]; +then + LOG_DELETE=$3 +fi + +echo "${COLOR}" +echo -------------------------------------------- +echo DURATION:$TEST_DURATIOMN_SEC +echo LOG INTERVAL:$LOG_INTERVAL_MIN +echo LOG_DELETE: $LOG_DELETE +echo -------------------------------------------- +echo "${NO_COLOR}" + +mkdir -p Report + +function RunLoadTest() +{ + #Params: + #$1 - Architecture (x86 or arm64).Used for logging and naming report file + #$2 - Stack output name to get API Url + #$3 - Stack output name to get lambda name + + #get test params from cloud formation output + echo "${COLOR}" + export API_URL=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$2'].OutputValue" \ + --output text) + echo API URL: $API_URL + + LAMBDA=$(aws cloudformation describe-stacks --stack-name $STACK_NAME \ + --query "Stacks[0].Outputs[?OutputKey=='$3'].OutputValue" \ + --output text) + echo LAMBDA: $LAMBDA + + if [ $LOG_DELETE == "yes" ]; + then + echo -------------------------------------------- + echo DELETING CLOUDWATCH LOG GROUP /aws/lambda/$LAMBDA + echo -------------------------------------------- + aws logs delete-log-group --log-group-name /aws/lambda/$LAMBDA + echo --------------------------------------------- + echo Waiting 10 sec. for deletion to complete + echo -------------------------------------------- + sleep 10 + fi + + #run load test with artillery + echo -------------------------------------------- + echo $1 RUNNING LOAD TEST $TEST_DURATIOMN_SEC sec $LAMBDA: $API_URL + echo -------------------------------------------- + echo "${NO_COLOR}" + artillery run \ + --overrides '{"config": { "phases": [{ "duration": '$TEST_DURATIOMN_SEC', "arrivalRate": 100 }] } }' \ + --quiet \ + ../../loadtest/load-test.yml + + echo "${COLOR}" + echo -------------------------------------------- + echo Waiting 10 sec. for logs to consolidate + echo -------------------------------------------- + sleep 10 + + #get stats from cloudwatch + enddate=$(date "+%s") + startdate=$(($enddate-($LOG_INTERVAL_MIN*60))) + echo -------------------------------------------- + echo Log start:$startdate end:$enddate + echo -------------------------------------------- + + QUERY_ID=$(aws logs start-query \ + --log-group-name /aws/lambda/$LAMBDA \ + --start-time $startdate \ + --end-time $enddate \ + --query-string 'filter @type="REPORT" | fields greatest(@initDuration, 0) + @duration as duration, ispresent(@initDuration) as coldstart | stats count(*) as count, pct(duration, 50) as p50, pct(duration, 90) as p90, pct(duration, 99) as p99, max(duration) as max by coldstart' \ + | jq -r '.queryId') + + echo -------------------------------------------- + echo Query started, id: $QUERY_ID + echo -------------------------------------------- + + echo --------------------------------------------- + echo Waiting 10 sec. for cloudwatch query to complete + echo -------------------------------------------- + sleep 10 + + echo -------------------------------------------- + echo RESULTS $LAMBDA + echo -------------------------------------------- + echo "${NO_COLOR}" + date > ./Report/load-test-report-$1.txt + echo $1 RESULTS lambda: $LAMBDA >> ./Report/load-test-report-$1.txt + echo Test duration sec: $TEST_DURATIOMN_SEC >> ./Report/load-test-report-$1.txt + echo Log interval min: $LOG_INTERVAL_MIN >> ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output text >> ./Report/load-test-report-$1.txt + cat ./Report/load-test-report-$1.txt + aws logs get-query-results --query-id $QUERY_ID --output json >> ./Report/load-test-report-$1.json +} + +RunLoadTest x86 ApiUrlX86 LambdaX86Name \ No newline at end of file diff --git a/src/NET8NativeMinimalAPI/template.yaml b/src/NET8NativeMinimalAPI/template.yaml index db2eb8e..181d59b 100644 --- a/src/NET8NativeMinimalAPI/template.yaml +++ b/src/NET8NativeMinimalAPI/template.yaml @@ -1,24 +1,28 @@ AWSTemplateFormatVersion: "2010-09-09" Transform: AWS::Serverless-2016-10-31 +Parameters: + x86FunctionName: + Type: String + Default: Net8-Native-MinimalApi-X86 + Globals: Function: MemorySize: 1024 - Architectures: ["x86_64"] Runtime: provided.al2 Timeout: 30 Tracing: Active Environment: Variables: PRODUCT_TABLE_NAME: !Ref Table - LOG_GROUP_PREFIX: !Sub "/aws/lambda/net-8-aot-minimal-" - LOAD_TEST_TYPE: "NET 8 native AOT Minimal API" Resources: - ApiFunction: + MinimalApiX86: Type: AWS::Serverless::Function Properties: - CodeUri: . + FunctionName: !Ref x86FunctionName + Architectures: [x86_64] + CodeUri: ./ Handler: ApiBootstrap Events: Api: @@ -30,18 +34,6 @@ Resources: - DynamoDBCrudPolicy: TableName: !Ref Table - - Version: "2012-10-17" - Statement: - - Sid: AllowStartQueries - Effect: Allow - Action: - - logs:DescribeLogGroups - - logs:StartQuery - Resource: "*" - - Sid: AllowGetQueryResults - Effect: Allow - Action: logs:GetQueryResults - Resource: "*" Metadata: BuildMethod: makefile @@ -55,8 +47,16 @@ Resources: KeySchema: - AttributeName: id KeyType: HASH + StreamSpecification: + StreamViewType: NEW_AND_OLD_IMAGES Outputs: ApiUrl: Description: "API Gateway endpoint URL" - Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/" \ No newline at end of file + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" + LambdaX86Name: + Description: "Lambda X86 Name" + Value: !Ref x86FunctionName + ApiUrlX86: + Description: "X86 API endpoint URL" + Value: !Sub "https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com" \ No newline at end of file