ZJIT: Run Ruby test suite

[tekknolagi]

We have our own little suite of tests, but we should start running the full Ruby test suite to discover all the weird little features we have broken.

[tekknolagi]

Oops, reopening to track full test suite

[st0012]

Fork PR for testing the configuration: #619

[st0012]

PR: ruby#13791

[st0012]

Result of running ZJIT against /test/ruby/test_*.rb as of today:

Passes

119/142 (84.4%)

Crashed (at least one of the test cases)

• test/ruby/test_array.rb
• test/ruby/test_encoding.rb
• test/ruby/test_env.rb
• test/ruby/test_fixnum.rb
• test/ruby/test_integer_comb.rb
• test/ruby/test_keyword.rb
• test/ruby/test_memory_view.rb
• test/ruby/test_pack.rb
• test/ruby/test_proc.rb
• test/ruby/test_ractor.rb
• test/ruby/test_rubyoptions.rb
• test/ruby/test_string.rb
• test/ruby/test_time.rb
• test/ruby/test_time_tz.rb
• test/ruby/test_variable.rb

Failed

• test/ruby/test_allocation.rb
• test/ruby/test_argf.rb
• test/ruby/test_bignum.rb
• test/ruby/test_parse.rb
• test/ruby/test_refinement.rb
• test/ruby/test_regexp.rb
• test/ruby/test_settracefunc.rb (intermittently)

Hang

• test/ruby/test_thread.rb

[tekknolagi]

Very cool. That's pretty good already. Do we have a way to categorize failures and crashes into groups?

[st0012]

I'm working on listing the problematic cases so we have a more detailed view and using the mechanism mentioned in ruby#13791 (comment) to skip them so we can run against /test/ruby/ on CI soon.

[st0012]

Finished through ruby#13791

[tekknolagi]

As discussed, let's make follow-up issues to describe the major things that block enabling the rest of the tests.