inspect.getsource returns unrelated code

[99991]

The documentation of inspect.getsource states:

Return the text of the source code for an object. [...]

But the function returns slightly more code. It returns all lines of source code that overlap the object, even if the rest of the lines belong to different objects.

For example, the following code creates two lambdas in the same line, so inspect.getsource will return the code of both lambdas instead of just one.

import inspect

# Two function declarations in the same line
x = lambda: 1; y = lambda: 2

# prints "x = lambda: 1; y = lambda: 2\n"
print(repr(inspect.getsource(y)))

The returned code is not even necessarily valid Python, as the following list comprehension demonstrates:

import inspect

funcs = [
    lambda: 1 for
    _ in range(3)
]

# prints '    lambda: 1 for\n', including indentation and a stray "for"
print(repr(inspect.getsource(funcs[0])))

Background

I was trying to access the abstract syntax tree of a function (for JIT compilation), but that does not appear to be possible for lambdas.

Linked PRs

• gh-143299: Doc: Clarify inspect.getsource behavior regarding shared lines #143301

[a12k]

I think this is a design limitation of the inspect module. The current doc, [r]eturn the text of the source code for an object, implies a 1:1 relationship between the object and source code. I think the fix is to update the docs to explicitly mention that inspect.getsource returns the lines of source code where the object is defined, but not necessarily valid Python and potentially adjacent code.

I can work on this. I'll update the documentation for inspect.getsource to clarify the above if there are no objections.

[picnixz]

@serhiy-storchaka Do you want to consider this a bug or not?

[picnixz]

I can work on this. I'll update the documentation for inspect.getsource to clarify the above if there are no objections.

@a12k In general, it's better to wait for a core developer to acknowledge the direction to take. It's not clear that this is only a limitation, it could really be an issue in some sense.

---

Now, more generally, if we document this, I think we should just say that we always return a full line and not a substring thereof. If the source code contains more stuff before or after but is still present on the same line, we won't strip it. Returning only the relevant portion may not be possible if we are lacking the column information (I don't remember if we lack it or not actually).

For instance:

def foo():
    def g():
        return 1
    return g

if we want the source of foo() (the returned value I mean), we'll get def g(): ... etc but with the indents. So this looks correct to me in this case that indents are kept.

[a12k]

Understood, thanks @picnixz. Should we head in the docs-changing direction, I've updated the gh-143301 to reflect your guidance.

[99991]

Returning only the relevant portion may not be possible if we are lacking the column information (I don't remember if we lack it or not actually).

I don't think that column information is easily obtainable.

getsource is a relatively thin wrapper around getsourcelines, which then calls findsource to fetch the relevant lines using e.g. func.__code__.co_firstlineno, so no column information used here.

The __code__ object of functions has co_lines member with two offsets in addition to line number, but those are offsets into the bytecode, not into the columns of the line.

The __code__ object also has a co_positions member, which looks like it might work, but I think it is broken because it returns column offset 0 for a function f which actually starts at column offset 24:

import inspect

lambda not_me: 666; f = lambda pick_me: 1; lambda me_neither: 666

# prints this:
# (3, 3, 0, 0) <---- (0, 0) is not the correct column offset :(
# (3, 3, 40, 41) <---- this is the offset of "1", which is correct
for t in f.__code__.co_positions():
    print(t)

(EDIT: I followed the trail starting at code_positionsiterator, which finally lead me to positionsiter_next, which returns the initial pi_column, which has been initialized to 0 instead of a properly decoded column position.)

[picnixz]

Well... then this becomes a question for the parser =/

cc @pablogsal

[pablogsal]

Well... then this becomes a question for the parser =/

cc @pablogsal

It's not (necessarily) the parser here is the compiler that sets the positions in the code objects ;)

[serhiy-storchaka]

#79037 may help for inspect.getsource(), but not for inspect.getsourcelines().

In theory, we may be able to find the starting position of the lambda, knowing the starting and ending position of its body. but it will be not easy. For example:

lambda x: lambda y=(lambda z: x(z)): x(y)