generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 32
feat(deploy-on-aws): add architecture diagram generation skill #84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MichaelWalker-git
wants to merge
10
commits into
awslabs:main
Choose a base branch
from
MichaelWalker-git:feat/diagram-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
f1007bb
feat(deploy-on-aws): add architecture diagram generation skill
MichaelWalker-git 4c4f700
fix(deploy-on-aws): address diagram skill review comments
MichaelWalker-git 3ea8a04
style(deploy-on-aws): fix dprint markdown table formatting
MichaelWalker-git ec41535
chore: trigger CI re-run
MichaelWalker-git c1c128c
chore: trigger contributor check
MichaelWalker-git 3d1f28c
fix(deploy-on-aws): fix remaining review comments on diagram skill
MichaelWalker-git 647be75
fix(deploy-on-aws): use generated-diagrams/ path in constructor example
MichaelWalker-git 95f00e8
fix(deploy-on-aws): address Copilot review comments on diagram skill
MichaelWalker-git 90cc2cf
fix(deploy-on-aws): address 3 new Copilot review comments
MichaelWalker-git 7172078
Merge branch 'main' into feat/diagram-skill
krokoko File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| --- | ||
| name: diagram | ||
| description: "Generate architecture diagrams as code using the Python diagrams DSL. Triggers on phrases like: architecture diagram, system design diagram, draw architecture, generate diagram, infrastructure diagram, AWS diagram, Kubernetes diagram, network diagram, visualize architecture." | ||
| --- | ||
|
|
||
| # Architecture Diagram Generation | ||
|
|
||
| Generate architecture diagrams using the Python [diagrams](https://diagrams.mingrammer.com/) package. Write a Python script, run it, and produce an image (PNG by default). | ||
|
|
||
| Supports AWS (29 service categories), Kubernetes, on-premises, GCP, SaaS, and custom icons via the `diagrams` DSL. | ||
|
|
||
| ## When to Load Reference Files | ||
|
|
||
| Load the appropriate reference file based on what the user is building: | ||
|
|
||
| - **Any diagram** -> ALWAYS load [references/dsl-syntax.md](references/dsl-syntax.md) first for DSL syntax, constructor options, connections, clusters, and edge styles | ||
| - **AWS architecture**, **cloud infrastructure**, or **AWS services** -> load [references/aws-services.md](references/aws-services.md) for AWS import paths, common icons, and 5 AWS examples | ||
| - **Kubernetes**, **on-premises**, **SaaS**, **flowcharts**, **process flows**, **custom icons**, or **non-AWS diagrams** -> load [references/non-aws-providers.md](references/non-aws-providers.md) for K8s, on-prem, flowchart, and custom icon examples | ||
|
|
||
| ## Workflow | ||
|
|
||
| 1. Load [references/dsl-syntax.md](references/dsl-syntax.md) | ||
| 2. Load the relevant provider reference (AWS or non-AWS) | ||
| 3. Write a Python script using the `diagrams` DSL | ||
| 4. Run: `mkdir -p generated-diagrams && python3 diagram.py` | ||
| 5. Verify the output file was created in `generated-diagrams/` | ||
|
|
||
| ## Critical Rules | ||
|
|
||
| - ALWAYS set `show=False` in `Diagram()` constructor | ||
| - ALWAYS set `filename="generated-diagrams/<name>"` (no `.png` extension) | ||
| - ALWAYS use explicit imports: `from diagrams.aws.compute import EC2` | ||
| - ALWAYS run `mkdir -p generated-diagrams` before executing | ||
| - Add `generated-diagrams/` to `.gitignore` to avoid committing output artifacts | ||
| - Use `Cluster()` to group related resources (VPCs, subnets, namespaces) | ||
| - Use `Edge(label="HTTPS", color="darkgreen", style="dashed")` for labeled or styled connections | ||
|
|
||
| ## Defaults | ||
|
|
||
| Default output format: PNG | ||
| Default layout direction: TB (top-to-bottom) | ||
|
|
||
| Override syntax: | ||
|
|
||
| - "left to right" -> Use `direction="LR"` | ||
| - "SVG format" -> Use `outformat="svg"` | ||
|
|
||
| When not specified, ALWAYS use `direction="TB"` and `outformat="png"`. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Requires two dependencies installed locally: | ||
|
|
||
| 1. **GraphViz** (system package providing `dot`): | ||
| - macOS: `brew install graphviz` | ||
| - Ubuntu/Debian: `sudo apt-get install graphviz` | ||
| - Amazon Linux/RHEL: `sudo yum install graphviz` | ||
| 2. **Python diagrams package**: `python3 -m pip install diagrams` | ||
|
|
||
| **Verify:** `dot -V && python3 -c "import diagrams; print('OK')"` | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Missing Dependencies | ||
|
|
||
| If `dot` command not found: | ||
|
|
||
| - Inform user: "GraphViz is not installed. Required for diagram rendering." | ||
| - Show install command for detected OS | ||
| - DO NOT attempt to generate diagrams without GraphViz | ||
|
|
||
| If `import diagrams` fails: | ||
|
|
||
| - Inform user: "Python diagrams package not installed." | ||
| - Show: `python3 -m pip install diagrams` | ||
| - DO NOT proceed without the package | ||
|
|
||
| ## References | ||
|
|
||
| - [DSL syntax and patterns](references/dsl-syntax.md) | ||
| - [AWS services and examples](references/aws-services.md) | ||
| - [Non-AWS providers and examples](references/non-aws-providers.md) | ||
| - [diagrams documentation](https://diagrams.mingrammer.com/) | ||
112 changes: 112 additions & 0 deletions
112
plugins/deploy-on-aws/skills/diagram/references/aws-services.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # AWS Services and Examples | ||
|
|
||
| ## AWS Service Categories | ||
|
|
||
| | Category | Import Path | Common Icons | | ||
| | ------------- | -------------------------- | ----------------------------------------------------- | | ||
| | `analytics` | `diagrams.aws.analytics` | Athena, EMR, Glue, Kinesis, Redshift, Quicksight | | ||
| | `compute` | `diagrams.aws.compute` | EC2, Lambda, ECS, EKS, Fargate, Batch | | ||
| | `database` | `diagrams.aws.database` | RDS, Aurora, DynamoDB, ElastiCache, Redshift, Neptune | | ||
| | `integration` | `diagrams.aws.integration` | SQS, SNS, StepFunctions, EventBridge, MQ | | ||
| | `management` | `diagrams.aws.management` | CloudWatch, CloudFormation, SystemsManager | | ||
| | `ml` | `diagrams.aws.ml` | Sagemaker, Rekognition, Comprehend, Bedrock | | ||
| | `network` | `diagrams.aws.network` | VPC, ELB, ALB, NLB, CloudFront, Route53, APIGateway | | ||
| | `security` | `diagrams.aws.security` | IAM, Cognito, WAF, KMS, Shield, SecretsManager | | ||
| | `storage` | `diagrams.aws.storage` | S3, EBS, EFS, FSx, Backup | | ||
| | `general` | `diagrams.aws.general` | User, Users, Client, InternetGateway | | ||
|
|
||
| Other categories: `ar`, `blockchain`, `business`, `cost`, `devtools`, `enablement`, `enduser`, `engagement`, `game`, `iot`, `media`, `migration`, `mobile`, `quantum`, `robotics`, `satellite`. | ||
|
|
||
| ## Example: Basic Web Service | ||
|
|
||
| ```python | ||
| from diagrams import Diagram | ||
| from diagrams.aws.compute import EC2 | ||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ELB | ||
|
|
||
| with Diagram("Web Service", show=False, filename="generated-diagrams/aws-basic"): | ||
| ELB("lb") >> EC2("web") >> RDS("userdb") | ||
| ``` | ||
|
|
||
| ## Example: Grouped Workers | ||
|
|
||
| ```python | ||
| from diagrams import Diagram | ||
| from diagrams.aws.compute import EC2 | ||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ELB | ||
|
|
||
| with Diagram("Grouped Workers", show=False, direction="TB", filename="generated-diagrams/aws-workers"): | ||
| ELB("lb") >> [EC2("w1"), EC2("w2"), EC2("w3"), EC2("w4"), EC2("w5")] >> RDS("events") | ||
| ``` | ||
|
|
||
| ## Example: Clustered Web Services | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster | ||
| from diagrams.aws.compute import ECS | ||
| from diagrams.aws.database import RDS, ElastiCache | ||
| from diagrams.aws.network import ELB, Route53 | ||
|
|
||
| with Diagram("Clustered Web Services", show=False, filename="generated-diagrams/aws-clustered"): | ||
| dns = Route53("dns") | ||
| lb = ELB("lb") | ||
| with Cluster("Services"): | ||
| svc_group = [ECS("web1"), ECS("web2"), ECS("web3")] | ||
| with Cluster("DB Cluster"): | ||
| db_primary = RDS("userdb") | ||
| db_replica = RDS("userdb ro") | ||
| db_primary - db_replica | ||
| memcached = ElastiCache("memcached") | ||
| dns >> lb >> svc_group | ||
| svc_group >> db_primary | ||
| svc_group >> memcached | ||
| ``` | ||
|
|
||
| ## Example: Event Processing | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster | ||
| from diagrams.aws.compute import ECS, EKS, Lambda | ||
| from diagrams.aws.analytics import Redshift | ||
| from diagrams.aws.integration import SQS | ||
| from diagrams.aws.storage import S3 | ||
|
|
||
| with Diagram("Event Processing", show=False, filename="generated-diagrams/aws-events"): | ||
| source = EKS("k8s source") | ||
| with Cluster("Event Flows"): | ||
| with Cluster("Event Workers"): | ||
| workers = [ECS("w1"), ECS("w2"), ECS("w3")] | ||
| queue = SQS("event queue") | ||
| with Cluster("Processing"): | ||
| handlers = [Lambda("p1"), Lambda("p2"), Lambda("p3")] | ||
| store = S3("events store") | ||
| dw = Redshift("analytics") | ||
| source >> workers >> queue >> handlers | ||
| handlers >> store | ||
| handlers >> dw | ||
| ``` | ||
|
|
||
| ## Example: S3 Image Processing with Bedrock | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster, Edge | ||
| from diagrams.aws.compute import Lambda | ||
| from diagrams.aws.general import User | ||
| from diagrams.aws.ml import Bedrock | ||
| from diagrams.aws.storage import S3 | ||
|
|
||
| with Diagram("S3 Image Processing", show=False, direction="LR", filename="generated-diagrams/aws-bedrock"): | ||
| user = User("User") | ||
| with Cluster("S3 Bucket"): | ||
| input_folder = S3("Input") | ||
| output_folder = S3("Output") | ||
| fn = Lambda("Processor") | ||
| bedrock = Bedrock("Claude Sonnet") | ||
| user >> Edge(label="Upload") >> input_folder | ||
| input_folder >> Edge(label="Trigger") >> fn | ||
| fn >> Edge(label="Process") >> bedrock | ||
| bedrock >> Edge(label="Result") >> fn | ||
| fn >> Edge(label="Save") >> output_folder | ||
| ``` |
110 changes: 110 additions & 0 deletions
110
plugins/deploy-on-aws/skills/diagram/references/dsl-syntax.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| # Diagrams DSL Syntax Reference | ||
|
|
||
| ## Basic Structure | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster, Edge | ||
| from diagrams.aws.compute import EC2 | ||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ALB | ||
|
|
||
| with Diagram("Title", show=False, filename="generated-diagrams/name"): | ||
| with Cluster("VPC"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private Subnet"): | ||
| servers = [EC2("web1"), EC2("web2")] | ||
| db = RDS("PostgreSQL") | ||
| lb >> servers >> db | ||
| ``` | ||
|
|
||
| ## Diagram Constructor | ||
|
|
||
| ```python | ||
| with Diagram( | ||
| "Diagram Title", # Title shown on the diagram | ||
| show=False, # ALWAYS False -- don't open viewer | ||
| filename="generated-diagrams/name", # Output path (no .png extension) | ||
| direction="TB", # TB (top-bottom), LR (left-right), BT, RL | ||
| outformat="png", # png (default), jpg, svg, pdf | ||
| ): | ||
| pass # Add nodes and connections here | ||
| ``` | ||
|
|
||
| ## Connections | ||
|
|
||
| ```python | ||
| node1 >> node2 >> node3 # Left to right flow | ||
| node1 << node2 # Right to left flow | ||
| node1 - node2 # Bidirectional | ||
| node1 >> [node2, node3, node4] # Fan out to multiple | ||
| node1 >> Edge(label="HTTPS", color="darkgreen", style="dashed") >> node2 | ||
| ``` | ||
|
|
||
| ## Clusters | ||
|
|
||
| ```python | ||
| with Cluster("VPC"): | ||
| with Cluster("Public Subnet"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private Subnet"): | ||
| app = [EC2("app1"), EC2("app2")] | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| with Cluster("Data"): | ||
| db = RDS("db") | ||
| lb >> app >> db | ||
| ``` | ||
|
|
||
| ## Edge Styles | ||
|
|
||
| | Parameter | Values | | ||
| | --------- | --------------------------------------------------------------------------------- | | ||
| | `color` | `"darkgreen"`, `"firebrick"`, `"brown"`, `"darkorange"`, `"black"`, any CSS color | | ||
| | `style` | `"solid"`, `"dashed"`, `"dotted"`, `"bold"` | | ||
| | `label` | Any string | | ||
|
|
||
| ## Provider Import Paths | ||
|
|
||
| | Provider | Import Pattern | Example | | ||
| | ----------- | --------------------------------- | --------------------------------------------------- | | ||
| | AWS | `diagrams.aws.<category>` | `from diagrams.aws.compute import EC2` | | ||
| | GCP | `diagrams.gcp.<category>` | `from diagrams.gcp.storage import GCS` | | ||
| | Kubernetes | `diagrams.k8s.<category>` | `from diagrams.k8s.compute import Pod` | | ||
| | On-premises | `diagrams.onprem.<category>` | `from diagrams.onprem.database import PostgreSQL` | | ||
| | SaaS | `diagrams.saas.<category>` | `from diagrams.saas.chat import Slack` | | ||
| | Programming | `diagrams.programming.<category>` | `from diagrams.programming.flowchart import Action` | | ||
| | Generic | `diagrams.generic.<category>` | `from diagrams.generic.compute import Rack` | | ||
| | Custom | `diagrams.custom` | `Custom("name", "icon.png")` | | ||
|
|
||
| ## Common Patterns | ||
|
|
||
| ### Fan-out / Fan-in | ||
|
|
||
| ```python | ||
| source >> [worker1, worker2, worker3] >> sink | ||
| ``` | ||
|
|
||
| ### Bidirectional with Replica | ||
|
|
||
| ```python | ||
| primary = RDS("primary") | ||
| primary - RDS("replica") | ||
| ``` | ||
|
|
||
| ### Nested Clusters | ||
|
|
||
| ```python | ||
| with Cluster("VPC"): | ||
| with Cluster("Public"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private"): | ||
| app = [EC2("app1"), EC2("app2")] | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| lb >> app | ||
| ``` | ||
|
|
||
| ### Custom Nodes | ||
|
|
||
| ```python | ||
| from diagrams.custom import Custom | ||
|
|
||
| # Place icon files in your project directory beforehand | ||
| custom_node = Custom("My Service", "./icons/my-service.png") | ||
| ``` | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.