Python Stack Tracer

🎯 TL;DR

Python Stack Tracer is a terminal-based visualization tool that shows how function calls, arguments, and local variables appear on the call stack as a Python program runs.

python-stack-tracer-demo-01.mov

🎯 What is Stack Tracer?

Have you wondered how function frames, arguments and local variables look on the stack? Or how recursion causes the stack to "overflow"?

Python Stack Tracer is a tool to help us visualize what the stack looks like as code executes, in the style of x86-64 calling convention.

This tool is meant to help people learn calling conventions and functions in memory, and build an intuitive mental model for learning C/C++ and memory exploits like buffer overflows. Python's stack is really just an abstraction, because its function frames actually live on the heap as a singly linked list. The real C-stack is invisible to us, and is what runs the Python interpreter.

🎯 Tech

• Python
• Rich (Python library for terminal formatting).

🎯 How to Run?

1. Clone this repo inside your chosen directory

git clone https://github.com/adawongHL/python-stack-tracer.git

2. Make sure you have the python interpreter installed
3. Create a venv and activate it

python -m venv venv

venv\Scripts\activate       # Windows
source venv/bin/activate    # Mac/Linux

4. Install project dependencies listed in requirements.txt

pip install -r requirements.txt

5. Execute the stack tracer

python3 main.py <your_program.py>

6. Press Enter to step through each line of code

🎯 Process

1. Started with the idea of building a tool that helps people understand memory layout
2. Decided on a stack tracer running locally (potentially converting to a web app in the future)
3. Researched terminal formatting tools and settled on Rich
4. Built the first version that prints to the terminal without Rich. Experimented with how to display the stack.
5. Built the second version with Rich, with 3 chunks in the UI:
   • Left chunk: currently executing line from the user’s code
   • Right chunk: stack layout
   • Bottom chunk: program output

🎯 What I Learnt

Building the stack tracer had me look under the hood of how Python works, and I walked away learning how Python runs functions and how its function frames work.

Python frames

• are all HEAP objects
• they exist as a singly linked list, where callee functions point to their caller using the reference f_back
• the actual C function call stack that runs the Python interpreter is invisible to us
  • Python interpreter is written in C
• however, we can use sys.settrace to trace Python's abstraction of the stack frames

How Rich works on a high level

• Create a Layout object. I cut it into left, right, bottom.
• Panels are renderable objects, among others
• Inside each chunk of my Layout, I placed Panels:
  • Panel for stack,
  • Panel for source code
  • Panel for stdout

Troubleshooting bugs

• When I tried to redirect program output (stdout) into an IO buffer, Rich also got redirected into this buffer, which caused crashes. What worked was forcing Rich to render into the original stdout.

🎯 Future Improvements

• User-supplied programs should be runnable from any folder and not need to be placed in the same directory
• When source code gets too long, make Rich "scroll" to keep active line in view
  • Same with the stdout output section
• Add toggle keys for modes:
  • comprehensive mode where you can see module import code (currently hide it because it distracts us from the core of the code), just so we can appreciate how much stuff goes on under the hood even for "simple" programs
• Add nuanced details: saved stack base pointer, registers , ret addr
• Migrate to web app so people can write Python code in a browser editor and run the code live, and see the stack animation
• Expand to other langs like C/C++
• Expand to other calling conventions e.g. ARM calling convention