Add files via upload

[Russellwangz]

Scrape from over 16 real Github repositories to find ec2 test cases with filters and configurations
For those multiple - layer chained cli uses multiple resources, split them up and made some refinement (Replace dependent cli with the independent cli/Remove duplicated & non-executable cli ).
Test against emulators
Label them ec2, non-ec2 (Services other than EC2), augmented ec2 (add more commands and strip off some commands for non-ec2 cli to target for ec2)

[zyang37]

Thanks for the PR, a few issues to address before I merge:

1. The format seems inconsistent with what's in the current ec2/.
2. For services other than ec2, please create separate directories. Ex: tests/cli/ssm/....

[Russellwangz]

Hi Zhen Ning,

These cli are scraped from real GitHub repos, therefore, they are just in shell script format rather than json or rst. All of them are regarding ec2 rescources as I augmented those if they use services other than ec2. So do I make a sub repo like tests/github-cli, I mean each of them are labelled with what they are testing and the operations like describe or create , so I think it is ok to put them together.

Thanks,
Russell

[zyang37]

Hi Russell,

No need to create a sub repo, simply create a new folder for other services. Something like: emulators/aws-ec2/tests/cli/sts/... parallel to ec2/.

The current structure is confusing because:

1. Test cases for other services are mixed with EC2s, and placed under ec2/.
2. Format being different from others, so the run test script on the high level won't detect these newly added cases.

To ensure these newly added case work, you could follow the test README to run through the test cases. And make sure your cases are detected and added to the tests. In the current stage, we don't support services other than ec2 yet, so you can just leave testcases for other services in their new folder for now.

Let me know if you have any questions. Thanks.

[Russellwangz]

Hi Zhen Ning,

The non-ec2 cli are augmented into ec2 AWS CLI, that is, using the equivalent EC2 CLI to do the same thing. I know it seems a little bit confusing as I kept those non-EC2 CLI to understand what it is originally doing. The augmented ones are what we really need. These test scripts are in shell script since most of them were in shell format when I scraped those, and they can be parsed and executed by the emulator using bash script like this
cd /Users/russell/vera/emulators/aws-ec2/tests
source ../.venv/bin/activate
python3 eval_emulator.py test_labeled.sh
I am wondering which repo we should place them as they are not rst format as other test files. I hope this makes it clear.

Thanks!