`PyArg_Parse`: Provide `N` as new format unit for non negative `Py_ssize_t`

[wiomoc]

Feature or enhancement

Proposal:

In gh-135763 / gh-138150 the Argument Clinic converter for Py_ssize_t was extended to optionally validate for values to be non-negative / >=0.
In principle, it would now also make sense to offer similar behavior/features for PyArg_ParseIn principle, it would now also make sense to offer similar behavior/features for PyArg_Parse*.
As n is the format unit for "signed" Py_ssize_t values, I propose N as the "unsigned" format unit for Py_ssize_t values.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

#138150 (comment)

Linked PRs

• gh-138361: Add N format unit for non-negative Py_ssize_t #138362

[AA-Turner]

Have there been prior requests for a feature like this? I'm trying to guage the need for it beyond our internal use-case.

A

[serhiy-storchaka]

N is already used in Py_BuildValue() with different semantic.

If add such option, we should consider more general feature applicable for all integer types. Currently, there are no formats for unsigned types -- format codes like I accept also negative integers.

[picnixz]

And note that u/U and z/Z were removed in 3.12 so I don't know if it's better to re-use them or not. Alternatively, we could have n> or m as we use n for size_t.

By the way, I is documented as accepting unsigned int only (in https://docs.python.org/3/c-api/arg.html#numbers). We do have format codes for unsigned types but there are some deprecation warnings that can be emitted in some cases.

Since we just continue the alphabet whenever we add a new format code, we can just use m for Py_ssize_t that is >= 0

[encukou]

I think this is easy enough to validate in user code; we don't need a new letter for it.

[vstinner]

I think this is easy enough to validate in user code; we don't need a new letter for it.

I agree. I don't think that we should add a "positive only" variant to all integer types.

[picnixz]

For Py_ssize_t, my rationale was as follows: sizes in Python are Py_ssize_t, not size_t, so the fields are Py_ssize_t (e.g., Py_buffer) and the parameters of some functions such as PyBytes_FromStringAndSize are also Py_ssize_t. OTOH, sizes in external C functions are usually size_t, so the compiler can complain if we don't cast to size_t. With AC, we previously needd:

/*[clinic input]
module.foo
  length: Py_ssize_t
[clinic start generated code]*/
static PyObject *
module_foo_impl(PyObject *module, Py_ssize_t length)
[...]
{
     if (length < 0) { raise(); }
     c_foo(..., (size_t)length);
     return ...;
}

Now we can do:

/*[clinic input]
module.foo
  length: Py_ssize_t(allow_negative=False)
[clinic start generated code]*/
static PyObject *
module_foo_impl(PyObject *module, Py_ssize_t length)
[...]
{
     assert(length >= 0);
     c_foo(..., (size_t)length);
     return ...;
}

And we eliminate lots of code duplication in the user code. Thus, I'd like to offer the same possibilities to regular users that don't have AC so that they don't need this additional logic.

But if you only want this to be internal for us, I'm ok (at least, I can now refactor the code and make it shorter).

[encukou]

Yes, CPython wraps a lot of wrap libc & POSIX APIs, perhaps often even without needing to check an upper bound. The AC support was a great addition.

But the question here is, do other users need it often enough?

[picnixz]

That I don't know. Considering many core devs expressed some doubts and that I myself don't necessarily need it, I would suggest we postpone this decision for the C API until someone asks for it (maybe we can make a DPO thread to gather some stats).

[vstinner]

@wiomoc closed the PR #138362.