The c code is compiled into a byte object that python then places into an exectuable and writeable mmapped block.
The c code is not linked, so can not use any external functions. It can however use the Python.h header for macros and struct field offsets.
Defines a few functions and uses the Python.h header.
Takes queuemodule.c, compiles it to queuemodule.o and copies the
.text section to a python bytes object in the file text.py.
I have to use objdump to manually see the offsets each function exists at.
objdump -M intel -d queuemodule.o
The dumped .text section is then copied into a python byte literal
in the actual queue.py program, and function offsets
are noted (keep hex in mind).
First ctypes is used to get access to the mmap function, which allows us to allocate a block in memory and mark it as executable (as well as writable, which is a big security no-no). This block will serve both as our text section (aka machinecode) as well as our data section at some offset.
Ctypes lets us cast random addresses to c functions with specific inputs and return types, and then call them. That's what we do.
However, ctypes is only used for initial calls to C (it's slow).
These calls are to a function that takes a PyObject* as input and changes what
function it refers to.
To be clear: Inside cpython lots of modules are written in c. When I import
these modules, the functions in them are of a type that more or less just
points to a c function. By passing these objects to my evil entrypoint function setCallAddress.
After the call addresses of the builtins have been changed, any call
to those functions will now lead to your C-code, but that code still
has to somewhat behave like python source. You get PyObject* as input,
and must return PyObject* as output.
Python is reference counted, so whenever you return something, make sure to also increase the reference count.
Py_INCREF(item);
return item;This of course also applies if you are going to be holding on to the object. As soon as you return it however, the caller owns the object, and they will decrease the reference count if you don't do anything with it on the python side.
You might think the solution is to return None, but alas:
- The global python None object is also reference counted (??)
- You don't even have access to it because you don't know the address (unless you pass it and keep it)
I couldn't find a way of storing data between function calls, so I ended up simply putting a pointer to the data section snugly inside python's own memory at an arbtrary offset from the module. The self object turns out to be the module and not the function, so I couldn't use the function object, take the function pointer and add an offset to get to my custom data section behind my custom text section.
It sucks, I have no idea what is supposed to be at id(module)+10*sizeof(size_t**).
While debugging, using hex(id(my_obj)) in python is great to recognize
objects in stack traces.
To even get stack traces I recommend going into a target function in
the cpython source code, e.g. m_atan2 in Modules/mathmodule.c, and adding
#include <signal.h>
static double
m_atan2(double y, double x)
{
raise(SIGINT);
....This will cause an interrupt whenever you try to calculate an atan2.
Use this by running in gdb, calling math.atan2 and upon SIGINT,
say bt in gdb to get a backtrace, which will also show what functions
were called to get here and, importantly, the parameters passed. Use the hex(id())
tick to recognize objects from the python side.
Note: this m_atan2 function is not one you can override. It is called by
a wrapper function that turns PyObject* into doubles and stuff.
The functions I managed to override always took and returned PyObject*,
or maybe a pair of PyObject** and size_t if you're lucky.
Because we don't link, we can only use macros (and static functions??). If we ever try to call an actual function from CPython from our c program, the object file will contain a placeholder address, which will of course never be fixed.
In my experience it is very hard to get the signature right when changing functions.
You need the same amount of args (even if you don't use the last ones).
You also need to know if you get a PyObject* or a pointer to an array of them,
or if the object you get is acutally a tuple of args.
For python 3.6.8 i had to find a module to "attack" with some functions of the correct type. You need to know if the built in function you override is a METH_O (for no args), METH_O (for one arg), or METH_VARARG (for more than one / arbitrary).
This has changed in python 3.8 I think. Know the version you are attacking!!
This is all to try and get a good time on the INGInious scoreboard in TDT4120.
The version of python is different, so I compile my own from the cpython repository to test.
Python version on server:
import sys
ver = sys.version_info
assert ver.major == 3
assert ver.minor == 6
assert ver.micro == 8