Perception dev

[hillhack]

environment_state work #71

Location

• Defined in: mesa-llm/mesa_llm/reasoning/reasoning.py
• Type: EnvironmentalState | None (optional field in Observation dataclass)
• Populated by: LLMAgent._get_environmental_state() in llm_agent.py

Structure (EnvironmentalState dataclass)

@dataclass
class EnvironmentalState:
    current_cell: dict[str, Any] | None = None        # Agent's current location data
    visible_cells: dict[str, dict[str, Any]] | None = None  # Nearby cells within vision
    statistics: dict[str, dict[str, float | int]] | None = None  # Aggregated stats
    global_environment: dict[str, Any] | None = None  # Model-level variables

Each Field Explained

1. current_cell - What the agent is standing on

{
    "position": "(5, 5)",           # Stringified coordinates
    "properties": {                  # PropertyLayer values at this cell
        "sugar": 8.0,
        "walkable": True
    },
    "objects": [                     # Non-agent entities at this location
        {"type": "Tree", "pickable": False},
        {"type": "Gold", "pickable": True, "value": 100}
    ],
    "navigable": True,               # Can agent move here?
    "agent_count": 1                 # Number of agents (including self)
}

2. visible_cells - What the agent can see nearby

{
    "(5, 6)": {                      # Stringified position key
        "distance": 1.0,             # Euclidean distance from agent
        "direction": "east",         # Cardinal direction
        "relative_bearing": 90.0,    # Degrees (0=North, 90=East)
        "properties": {"sugar": 10.0},
        "objects": [],
        "navigable": True,
        "agent_count": 0
    },
    "(6, 5)": {                      # Another visible cell
        "distance": 1.0,
        "direction": "south",
        "relative_bearing": 180.0,
        ...
    }
}

3. statistics - Aggregated data across all visible cells

{
    "sugar": {
        "max": 10.0,                 # Maximum sugar in visible area
        "min": 2.0,                  # Minimum sugar in visible area
        "avg": 6.5,                  # Average sugar
        "count": 5                   # Number of cells with this property
    },
    "agent_count": {
        "total_visible": 3           # Total agents in visible area
    }
}

4. global_environment - Model-level state

{
    "weather": "sunny",
    "temperature": 25.5,
    "time_of_day": "morning",
    "season": "spring",
    "market_price": 100.0
}

How It's Generated

The flow in LLMAgent:

generate_obs()
  ├── _get_agent_location()          # Get (x, y) position
  ├── _get_local_state()             # Get neighboring agents
  └── _get_environmental_state()     # ← Creates EnvironmentalState
        ├── _get_current_cell_data()     # Current cell info
        ├── _get_visible_cells()         # Nearby cells (limited by max_visible_cells)
        │     ├── _get_cells_in_radius()    # Calculate positions in vision radius
        │     ├── _get_cell_properties()    # Read PropertyLayer values
        │     ├── _get_cell_objects()       # Detect non-agent objects
        │     ├── _is_cell_navigable()      # Check walkability
        │     └── _calculate_spatial_info() # Distance, direction, bearing
        ├── _calculate_statistics()      # Min/max/avg across visible cells
        └── _get_global_environment()    # Read model attributes

Configuration

Controlled by these LLMAgent parameters:

Parameter	Default	Effect
perceive_environment	True	Master switch - if False, environment_state is None
perceive_objects	False	Filter: False, "all", "pickable", "static"
max_visible_cells	10	Limits cells in visible_cells dict
global_env_attributes	List of attrs	Which model attributes to include
vision	None	Radius for neighbor and cell detection

Usage in Reasoning

The environment_state is passed to reasoning methods:

# In agent's step() method
observation = self.generate_obs()  # Contains environment_state
plan = self.reasoning.plan(
    prompt="What should I do?",
    obs=observation,  # Includes environment_state
    selected_tools=["move", "harvest"]
)

Reasoning classes (CoT, ReAct, ReWOO) can access:

• obs.environment_state.current_cell - Where am I?
• obs.environment_state.visible_cells - What's around me?
• obs.environment_state.statistics - What's the best direction?
• obs.environment_state.global_environment - What's the world state?

@colinfrisch i m still working on it. not completed yet.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[colinfrisch]

Great, thanks for your work, it's really something that we would be interested in integrating. As Ewout said, since there is a lot of developpement on space in mesa right now, things may change (also in mesa-llm integration). So put the effort more into the thinking than the implementation since we might need to change thing soon ;)

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 78 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.56%. Comparing base (6f9624d) to head (4b7bccc).

Files with missing lines	Patch %	Lines
mesa_llm/llm_agent.py	65.63%	78 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #71      +/-   ##
==========================================
- Coverage   90.03%   85.56%   -4.47%     
==========================================
  Files          19       19              
  Lines        1435     1635     +200     
==========================================
+ Hits         1292     1399     +107     
- Misses        143      236      +93     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.