Install matrix-specific pytest after test-requirements so pre-release job actually tests pre-release

Install matrix-specific pytest after test-requirements so pre-release job actually tests pre-release #9595