feat: Add device generation tools and validation workflow for TinyGo v0.39.0

[luoliwoshang]

Summary

This PR adapts the device/ directory to TinyGo v0.39.0 and implements LLGO compatibility. The device package contains two types of files:

1. Auto-generated files - Generated by code generation tools in _tools/ from upstream SVD/ATDF files
2. Manually maintained files - Architecture-specific files containing LLGO adaptation code

Reference: TinyGo v0.39.0 GNUmakefile

---

Device Generation Tools

_tools/gen-device-avr

• Purpose: Generate device definition code for AVR series chips
• Input: ATDF (Atmel Device File) files
• Output: device/avr/*.go
• Upstream: https://github.com/avr-rust/avr-mcu

_tools/gen-device-svd

• Purpose: Generate device definition code for ARM/RISC-V and other chip series
• Input: CMSIS-SVD (System View Description) XML files
• Output: device/{esp,nrf,nxp,sam,sifive,kendryte,stm32,rp,renesas}/*.go
• Upstream sources:
  • https://github.com/cmsis-svd/cmsis-svd-data
  • https://github.com/NordicSemiconductor/nrfx
  • https://github.com/tinygo-org/stm32-svd

---

Sync Workflow

Automated Generation Script

# Generate mode: Regenerate all device code from upstream
bash .github/scripts/device-gen.sh generate

# Verify mode: Check if existing code matches upstream
bash .github/scripts/device-gen.sh verify

Manual Execution

# 1. Install generation tools
go install ./_tools/gen-device-avr
go install ./_tools/gen-device-svd

# 2. Clone upstream data sources
git clone https://github.com/avr-rust/avr-mcu lib/avr
cd lib/avr && git checkout <specified-version> && cd ../..

# 3. Execute generation
gen-device-avr lib/avr/packs/atmega device/avr/
gen-device-avr lib/avr/packs/tiny device/avr/

# 4. Format code
go fmt ./device/avr/...

---

Preserved Files (Not Overwritten by Generation Script)

The following files are manually maintained:

• device/avr/avr.go, device/avr/avr_tiny85.go
• device/nrf/README.markdown
• device/nxp/hardfault.go, device/nxp/mimxrt1062_*.go
• device/sam/atsamd51x-bitfields.go, device/sam/atsame5x-bitfields.go
• device/rp/rp2040-extra.go, device/rp/rp2350-extra.go

---

LLGO Adaptation

TinyGo uses compiler intrinsics - functions without bodies that the compiler directly generates machine code for. LLGO doesn't recognize these intrinsics, so manual adaptation is required.

1. Inline Assembly Functions (Asm/AsmFull)

TinyGo syntax (no function body):

func Asm(asm string)
func AsmFull(asm string, regs map[string]interface{}) uintptr

LLGO adaptation (using //llgo:link directive):

//llgo:link Asm llgo.asm
func Asm(asm string) {}

//llgo:link AsmFull llgo.asm
func AsmFull(asm string, regs map[string]interface{}) uintptr { return 0 }

Note: Both Asm and AsmFull link to llgo.asm. LLGO automatically selects the corresponding behavior based on function signature.

Affected files:

• device/asm.go
• device/arm/arm.go
• device/arm64/arm64.go
• device/riscv/riscv.go
• device/avr/avr.go

2. SVCall System Calls (ARM Cortex-M)

LLGO doesn't support these yet, using panic placeholder:

func SVCall0(num uintptr) uintptr {
    panic("TODO: SVCall0")
}

Affected files: device/arm/arm.go, device/arm64/arm64.go

3. CSR Operations (RISC-V)

LLGO doesn't support these yet, using panic placeholder:

func (csr CSR) Get() uintptr {
    panic("TODO: CSR.Get()")
}

Affected files: device/riscv/csr.go (Get, Set, SetBits, ClearBits)

4. Interrupt Control Functions

TinyGo syntax (using //export):

//export DisableInterrupts
func DisableInterrupts() uintptr

LLGO adaptation (using //go:linkname):

//go:linkname DisableInterrupts DisableInterrupts
func DisableInterrupts() uintptr

Affected files: device/arm/arm.go

Note: device/riscv/riscv.go interrupt functions already have implementations (using CSR operations), no modification needed.

5. Import Path Standardization

TinyGo uses relative paths, which need to be changed to full module paths:

TinyGo	goplus/emb
"runtime/volatile"	"github.com/goplus/emb/runtime/volatile"
"device/arm"	"github.com/goplus/emb/device/arm"

6. No import "C" Required

device/arm/arm.go must NOT contain import "C", otherwise it causes compilation errors:

undefined: SCS_BASE
undefined: Asm

The C/assembly files are already stored in LLGO at /llgo/targets/device/.

---

Removed Redundant Files

The following .c and .S files have been removed as they are already in LLGO:

• device/arm/cortexm.S
• device/arm/interrupts.c
• device/riscv/handleinterrupt.S
• device/riscv/start.S

---

Directory Structure

goplus-emb/
├── _tools/
│   ├── gen-device-avr/     # AVR device generation tool
│   └── gen-device-svd/     # SVD device generation tool
├── .github/scripts/
│   └── device-gen.sh       # Automated generation script
├── device/
│   ├── asm.go              # Common inline assembly (manually maintained)
│   ├── arm/                # ARM architecture (partially manually maintained)
│   ├── arm64/              # ARM64 architecture (partially manually maintained)
│   ├── riscv/              # RISC-V architecture (partially manually maintained)
│   ├── avr/                # AVR device definitions
│   ├── esp/                # ESP device definitions
│   ├── nrf/                # Nordic device definitions
│   ├── stm32/              # STM32 device definitions
│   └── ...                 # Other devices
└── lib/                    # Temporary directory for upstream repos (not committed)

---

CI/CD

• Added .github/workflows/validate-embed-packages.yml for automated validation
• Added .github/actions/setup-llgo for LLGO environment setup
• LLGO version: 2d80951e7d8f5823c1fa92c8a7cb5dd351a10e6a

---

Change Statistics

• 1326 files changed
• +6,468 insertions
• -87,647 deletions (mainly removing .s assembly files)

[gemini-code-assist]

Summary of Changes

Hello @luoliwoshang, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the project's embedded development infrastructure by introducing robust tools and workflows for automated device package generation and validation. It adapts the system for TinyGo v0.39.0, ensuring that device-specific code for various microcontrollers can be consistently generated, compiled, and kept in sync with upstream sources, thereby improving the reliability and maintainability of embedded targets.

Highlights

• Automated LLGO Setup: A new GitHub Action has been introduced to streamline the setup and building of the LLGO compiler from a specific commit hash within CI workflows.
• Device Generation Tools: Two new Go-based tools, gen-device-avr and gen-device-svd, have been added to automate the generation of device-specific Go code for various microcontroller architectures (AVR, ARM-based via SVD).
• Device Validation Scripts: New shell scripts (device-gen.sh and validate_embed_packages.sh) are included to manage the generation of device packages and validate their compilation using LLGO, ensuring correctness and compatibility.
• CI Workflow for Validation: A validate-embed-packages CI workflow has been added to automatically test the generated device and machine packages, integrating the new validation scripts into the continuous integration process.
• Import Path Migration: All internal import paths have been updated from github.com/goplus/lib/emb to github.com/goplus/emb to reflect a new module structure.

Ignored Files

• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/validate-embed-packages.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive set of tools and CI workflows for device code generation and validation, migrated from another repository. The changes are well-organized and the scripts are quite detailed. My review focuses on improving the robustness and maintainability of the new tooling. I've identified several areas where error handling can be improved in both the shell scripts and Go generator tools to prevent silent failures and make the tools more resilient to unexpected inputs. Specifically, I've suggested changes to fail CI jobs on setup errors, handle filenames with spaces correctly, and replace panic calls in the Go tools with proper error returns.

[gemini-code-assist]

The use of panic for an unhandled register size makes the tool brittle. If an ATDF file contains an unexpected register size, the entire tool will crash. It's better to handle this case by returning an error, which can then be gracefully handled by the caller. This makes the tool more robust and provides better feedback to the user.

case 2:
				addReg(&Register{
					Name:    regEl.Name + "L",
					Type:    "volatile.Register8",
					Caption: regEl.Caption + " (lower bits)",
					Offset:  regOffset + 0,
				})
				addReg(&Register{
					Name:    regEl.Name + "H",
					Type:    "volatile.Register8",
					Caption: regEl.Caption + " (upper bits)",
					Offset:  regOffset + 1,
				})
			default:
				return nil, fmt.Errorf("unsupported register size %d for register %s", regEl.Size, regEl.Name)

[gemini-code-assist]

This file contains numerous panic() calls for error handling (e.g., on lines 209, 267, 574, 736, 793, etc.). Using panic for recoverable errors like parsing issues makes the tool brittle and difficult to reuse as a library. It's idiomatic in Go to return errors and handle them gracefully in a higher-level function, like main or generate. This would likely require changing several function signatures to return an error type alongside their normal return values.

[gemini-code-assist]

The llgo version verification step currently echoes a message on failure but allows the action to continue. This could mask installation or setup problems. To ensure the integrity of the setup, this step should fail if llgo is not found or fails to report its version.

        llgo version

[gemini-code-assist]

The ignore_list variable is treated as a space-separated string, which will cause issues if any of the filenames in the list contain spaces. It's more robust to define ignore_list as a bash array in the declare_*_config functions and iterate over it correctly.

For example, in declare_avr_config:

ignore_list=("avr.go" "avr_tiny85.go")

And then in generate_devices, you should iterate over it like this:

for file in "${ignore_list[@]}"; do

This change should be applied to all declare_*_config functions and all loops that iterate over ignore_list (lines 237, 268, 279). Also, checks for non-empty list should be changed from if [ -n "$ignore_list" ] to if [ ${#ignore_list[@]} -gt 0 ] (lines 235, 261).

[gemini-code-assist]

This TODO indicates that deriving enumerated values from other registers is not fully implemented. While it currently prints a warning, this means the tool will not generate correct code for SVD files that use this feature. This should be implemented to ensure correctness for all supported SVD files.

[fennoai]

Security: Path Traversal Vulnerability

Direct use of command-line arguments without validation could allow path traversal attacks (e.g., ../../etc/passwd).

Recommendation:

import "path/filepath"

func main() {
    if len(os.Args) < 3 {
        fmt.Fprintf(os.Stderr, "Usage: %s <input-dir> <output-dir>\n", os.Args[0])
        os.Exit(1)
    }
    
    indir, err := filepath.Abs(os.Args[1])
    if err != nil {
        fmt.Fprintf(os.Stderr, "Invalid input directory: %v\n", err)
        os.Exit(1)
    }
    
    outdir, err := filepath.Abs(os.Args[2])
    if err != nil {
        fmt.Fprintf(os.Stderr, "Invalid output directory: %v\n", err)
        os.Exit(1)
    }

[fennoai]

Security: XML External Entity (XXE) Vulnerability

XML decoder doesn't disable external entity processing, which could lead to information disclosure or SSRF if malicious ATDF files are processed.

Recommendation:

decoder := xml.NewDecoder(f)
decoder.Entity = xml.HTMLEntity // Limit to HTML entities only

[fennoai]

Security: Path Traversal Vulnerability

Same issue as gen-device-avr - command-line arguments should be validated and canonicalized before use to prevent path traversal attacks.

[fennoai]

Security: XML External Entity (XXE) Vulnerability

Same XXE vulnerability as in gen-device-avr. Disable external entity processing:

decoder := xml.NewDecoder(f)
decoder.Entity = xml.HTMLEntity

[fennoai]

Performance: Sequential Git Operations (Critical Bottleneck)

Git clone operations run sequentially for all 10 devices, wasting 100-600 seconds. Additionally, 7 devices share the same cmsis-svd-data repository but clone it separately (~350MB+ redundant transfers).

Recommendations:

1. Use shallow clones: git clone --depth 1
2. Share repository clones between devices that use the same repo
3. Parallelize device generation with background jobs

[fennoai]

Security: Potential Command Injection

The $task variable is expanded without proper quoting. While the current hardcoded values are safe, this pattern is risky.

Recommendation:
Use proper quoting or restructure to avoid word splitting:

"$generator" "$task" "$target_dir/"

[fennoai]

Code Quality: Unsafe Glob Deletion

If $target_dir is empty or manipulated, this could delete unintended files.

Recommendation:

if [ -z "$target_dir" ] || [ "$target_dir" = "/" ]; then
    echo "[$name] Error: Invalid target directory"
    return 1
fi
rm -rf "${target_dir:?}"/*

[fennoai]

Code Quality: Error Handling Issue

Only the first error is captured; subsequent errors are silently dropped. Also, wg.Done() should be called after error handling to ensure proper synchronization.

Recommendation:

var errMu sync.Mutex
var firstErr error

go func() {
    for filepath := range workChan {
        err := processFile(filepath, outdir)
        if err != nil {
            errMu.Lock()
            if firstErr == nil {
                firstErr = err
                fmt.Fprintf(os.Stderr, "Error processing %s: %v\n", filepath, err)
            }
            errMu.Unlock()
        }
        wg.Done()
    }
}()

[fennoai]

Documentation: Missing Script Header

Add documentation explaining the script's purpose, usage, and modes:

#!/bin/bash
#
# device-gen.sh - Generate device files from upstream hardware description files
#
# Usage: ./device-gen.sh [generate|verify]
#
# Modes:
#   generate - Clean target directories and regenerate all device files
#   verify   - Verify generated files match existing files (for CI)
#   (default) - Generate without cleaning (preserves ignore list files)
#
set -e

[fennoai]

Documentation: Missing Package Documentation

Add package-level documentation:

// gen-device-avr generates Go device packages from Atmel AVR ATDF descriptor files.
//
// This tool reads ATDF (Atmel Tools Device Files) XML files and generates:
//   - Go register definitions
//   - Interrupt vector definitions  
//   - Peripheral type definitions
//
// Usage:
//   gen-device-avr <input-dir> <output-dir>
//
// The tool processes ATDF files in parallel using all available CPU cores.
package main

[fennoai]

Documentation: Missing Package Documentation

Add package-level documentation:

// gen-device-svd generates Go device packages from ARM CMSIS-SVD descriptor files.
//
// Usage:
//   gen-device-svd [flags] <input-dir> <output-dir>
//
// Flags:
//   -source string       Source URL for SVD files (default "<unknown>")
//   -interrupts string   Interrupt system: "hardware" or "software" (default "hardware")
package main

[fennoai]

Code Review Summary

This PR adds valuable device generation infrastructure. The code is functional and well-structured, but several security vulnerabilities and performance bottlenecks need attention.

Critical Issues

Security (High Priority):

• Path traversal vulnerabilities in both generation tools - validate input paths before use
• XXE vulnerabilities in XML parsing - disable external entity processing
• Potential command injection in shell scripts - improve argument quoting

Performance (70-75% potential improvement):

• Sequential git operations waste 100-600s - use shallow clones and parallelize
• 7 devices redundantly clone the same cmsis-svd-data repo (~350MB+ wasted)
• SVD generation lacks parallelization (could be 5x faster)

Additional Improvements

Documentation: Missing package-level docs and script usage instructions make maintenance harder.

Error Handling: Race conditions in error channels, missing bounds checks, and panics instead of proper error returns.

See inline comments for specific recommendations and remediation code.