Blackbox test failure overview on master

[lordpretzel]

Keeping track of current state of black-box test failures and successes as reference and for bookkeeping. For now this is on postgres only. Add other backend results later.

Postgres

Setting noopt (heuristic optimization rules disabled)

Successful

• reg
• prov
• prov_use_tid
• prov_composable
• prov_composable_use_tid
• query_prov
• reenact
• dl.reg, dl.lineage

Failed

• temporal
  • temporal.seq [45/50]
    • temporal.seq.agg.q9
    • temporal.seq.nesting.q2
    • temporal.seq.nesting.q3
    • temporal.seq.nesting.q8
    • temporal.seq.cte.q3
    • temporal.seq.set.q2
• temporal.coalesce [35/38]
  • temporal.coalesce.agg.q9
  • temporal.coalesce.cte.q3
  • temporal.coalesce.set.q2
• dl.why

Setting optnopullnoselmove (heuristic optimization rules on (except for prov proj pullup and selection movearound)

Successful

• reg
• prov
• prov_use_tid
• prov_composable
• prov_composable_use_tid
• query_prov
• reenact
• dl.reg, dl.lineage

Failed

• dl.why
• temporal.seq
  • temporal.seq.cte
    • temporal.seq.cte.q3
    • temporal.seq.cte.q4
• temporal.seq.nesting
  • temporal.seq.set.q2
• temporal.seq.agg
  • temporal.seq.agg.q3
  • temporal.seq.agg.q5
  • temporal.seq.agg.q6
  • temporal.seq.agg.q9
  • temporal.seq.agg.q11
• temporal.seq.subquery
  • temporal.seq.subquery.q2
  • temporal.seq.subquery.q3

Setting heuopt (heuristic optimization rules on)

Successful

Failed

DuckDB

Setting noopt (heuristic optimization rules disabled)

Successful

• reg
• prov
• prov_use_tid
• prov_composable
• prov_composable_use_tid
• query_prov
• reenact
• semiring
• dl.reg, dl.lineage

Failed

• dl.why
• temporal
  • temporal.seq [45/50]
    • temporal.seq.agg.q9
    • temporal.seq.nesting.q2
    • temporal.seq.nesting.q3
    • temporal.seq.nesting.q8
    • temporal.seq.cte.q3
    • temporal.seq.set.q2
• temporal.coalesce [35/38]
  • temporal.coalesce.agg.q9
  • temporal.coalesce.cte.q3
  • temporal.coalesce.set.q2

Setting optnopullnoselmove (heuristic optimization rules on (except for prov proj pullup and selection movearound)

Successful

• reg
• prov
• prov_use_tid
• prov_composable
• prov_composable_use_tid
• reenact
• dl.reg, dl.lineage

Failed

• dl.why
• temporal.seq
  • temporal.seq.cte
    • temporal.seq.cte.q3
    • temporal.seq.cte.q4
• temporal.seq.nesting
  • temporal.seq.set.q2
• temporal.seq.agg
  • temporal.seq.agg.q3
  • temporal.seq.agg.q5
  • temporal.seq.agg.q6
  • temporal.seq.agg.q9
  • temporal.seq.agg.q11
• temporal.seq.subquery
  • temporal.seq.subquery.q2
  • temporal.seq.subquery.q3

Setting heuopt (heuristic optimization rules on)

Successful

Failed