Skip to content

Fix xeus-octave kernel startup#80

Merged
rgbkrk merged 1 commit into
mainfrom
quill/add-xeus-octave
Mar 31, 2026
Merged

Fix xeus-octave kernel startup#80
rgbkrk merged 1 commit into
mainfrom
quill/add-xeus-octave

Conversation

@rgbkrk
Copy link
Copy Markdown
Member

@rgbkrk rgbkrk commented Mar 31, 2026

Summary

  • Set OCTAVE_HOME and OCTAVE_EXEC_PREFIX env vars in CI so xeus-octave can find Octave's built-in .m scripts (the root cause was graphics_toolkit not being found, causing the kernel to crash with SIGABRT)
  • Increase per-test timeout to 30s for xeus-octave in CI (Octave runtime is slow to initialize)
  • Detect kernel process crash before attempting ZMQ connections, with stderr capture for diagnostics
  • Retry kernel_info_request up to 3 times with 2s delays for slow-starting kernels
  • Increase initial kernel startup delay from 500ms to 2s

Results

xeus-octave now starts and passes conformance tests (15-23/27 depending on run, some flakiness in interactive tests). Previously it crashed immediately with:

terminate called after throwing an instance of 'octave::execution_exception'
  what():  feval: function 'graphics_toolkit' not found

Test plan

  • xeus-octave CI job starts kernel successfully
  • xeus-octave passes Tier 1 basic protocol tests
  • All other kernel jobs remain unaffected

@github-actions
Copy link
Copy Markdown

Kernel Conformance Matrix

Kernel Protocol Tier 1 Tier 2 Tier 3 Tier 4 Total
almond 5.4 6/11 4/7 1/4 1/5 12/27
ark 5.4 11/11 7/7 3/4 5/5 26/27
async-kernel 5.5 11/11 7/7 4/4 5/5 27/27
deno 5.3 10/11 7/7 4/4 5/5 26/27
evcxr 5.3 10/11 3/7 2/4 3/5 18/27
gonb 5.4 6/11 1/7 1/4 3/5 11/27
ijulia 5.4 10/11 6/7 3/4 5/5 24/27
ipykernel 5.3 11/11 7/7 4/4 5/5 27/27
ocaml-jupyter 5.2 9/11 4/7 2/4 1/5 16/27
xeus-cling 5.3 9/11 4/7 0/4 0/5 13/27
xeus-cpp 5.6 5/11 0/7 0/4 0/5 5/27
xeus-haskell 5.6 7/11 5/7 1/4 2/5 15/27
xeus-lua 5.6 10/11 6/7 1/4 5/5 22/27
xeus-octave unknown 0/1 0/0 0/0 0/0 0/1
xeus-python 5.6 11/11 7/7 4/4 5/5 27/27
xeus-r 5.6 11/11 7/7 3/4 5/5 26/27
xeus-sql 5.6 8/11 5/7 0/4 4/5 17/27
xeus-sqlite 5.6 8/11 6/7 0/4 4/5 18/27

almond (scala) - 15 failures, 0 skipped

Language: scala | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_reply_ok execute_request harness_error Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
status_busy_idle_lifecycle status unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
execute_input_broadcast execute_input unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field history
error_handling execute_request unknown Protocol error: Error deserializing content for msg_type error: missing field traceback
execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mid\u{1b}[39m: \u{1b}[32mString\u{1b}[39m = \u{1b}[32m"a7b2e5a4-a5cd-49ab-9841-7c031a222bc7"\u{1b}[39m", expected internally tagged enum ExpressionResult
rich_execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mres11\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m2\u{1b}[39m", expected internally tagged enum ExpressionResult
stdin_input_request input_request harness_error Timeout waiting for iopub idle (stdin test)
comms_lifecycle comm_open harness_error Kernel unresponsive after comm: Timeout waiting for iopub idle
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status
execution_count_increments execute_request unknown Counts didn't increment: 12 -> 0
shutdown_reply shutdown_request unknown Protocol error: Error deserializing content for msg_type shutdown_reply: missing field status
ark () - 1 failures, 0 skipped

Language: R | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
deno (Deno kernel) - 1 failures, 0 skipped

Language: typescript | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
evcxr (evcxr_jupyter) - 6 failures, 3 skipped

Language: Rust | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • stdin_input_request: Kernel can request input from frontend via stdin channel
gonb (gonb) - 15 failures, 1 skipped

Language: go | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
complete_request complete_request unknown Timeout waiting for shell reply
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
is_complete_incomplete is_complete_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
ijulia (ijulia) - 2 failures, 1 skipped

Language: julia | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
comm_info_request comm_info_request unknown Timeout waiting for shell reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
ocaml-jupyter (ocaml-jupyter) - 9 failures, 2 skipped

Language: OCaml | Protocol: 5.2

Failures

Test Message Type Likely Source Reason
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field status
comm_info_request comm_info_request unknown Timeout waiting for shell reply
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Timeout waiting for iopub idle
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-cling (xeus-cling) - 12 failures, 2 skipped

Language: c++ | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown inspect_reply status: Error
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: data did not match any variant of untagged enum HistoryEntry
comm_info_request comm_info_request unknown Timeout waiting for shell reply
display_data display_data unknown Timeout waiting for iopub idle
update_display_data update_display_data unknown Timeout waiting for iopub idle
execute_result execute_result unknown Protocol error: Codec Error: Broken pipe (os error 32)
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-cpp (xeus-cpp) - 20 failures, 2 skipped

Language: C++ | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
execute_stderr stream unknown Timeout waiting for iopub idle
execute_reply_ok execute_request harness_error Protocol error: Codec Error: Broken pipe (os error 32)
status_busy_idle_lifecycle status unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_input_broadcast execute_input unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
complete_request complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
inspect_request inspect_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_complete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_incomplete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
history_request history_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comm_info_request comm_info_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
error_handling execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
display_data display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
update_display_data update_display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_result execute_result unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-haskell (xhaskell) - 8 failures, 4 skipped

Language: haskell | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Protocol error: Codec Error: Broken pipe (os error 32)
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-lua (xlua) - 3 failures, 2 skipped

Language: lua | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stderr stream unknown No stderr containing 'error'
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-octave (unknown) - 1 failures, 0 skipped

Language: Octave | Protocol: unknown

Failures

Test Message Type Likely Source Reason
kernel_startup kernel_info_request protocol_error Protocol error: Codec Error: Broken pipe (os error 32)
xeus-r (xr) - 1 failures, 0 skipped

Language: R | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
xeus-sql (xsql) - 7 failures, 3 skipped

Language: MySQL | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-sqlite (xsqlite) - 5 failures, 4 skipped

Language: sql | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel

Test Results by Message Type

Test Message Type almond ark async-kernel deno evcxr gonb ijulia ipykernel ocaml-jupyter xeus-cling xeus-cpp xeus-haskell xeus-lua xeus-octave xeus-python xeus-r xeus-sql xeus-sqlite
comm_info_request comm_info_request -
comms_lifecycle comm_open -
complete_request complete_request -
display_data display_data ⏭️ ⏭️ - ⏭️ ⏭️
error_handling execute_request -
execute_input_broadcast execute_input -
execute_reply_ok execute_request -
execute_result execute_result -
execute_stderr stream -
execute_stdout execute_request -
execution_count_increments execute_request -
heartbeat_responds heartbeat -
history_request history_request -
inspect_request inspect_request -
interrupt_request interrupt_request -
iopub_welcome iopub_welcome ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ -
is_complete_complete is_complete_request ⚠️ - ⚠️
is_complete_incomplete is_complete_request ⚠️ ⚠️ ⚠️ ⚠️ - ⚠️ ⚠️
kernel_info_has_language_info kernel_info_request -
kernel_info_has_protocol_version kernel_info_request -
kernel_info_reply_valid kernel_info_request -
kernel_startup kernel_info_request - - - - - - - - - - - - - - - - -
parent_header_correlation parent_header -
rich_execute_result execute_result ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ - ⏭️ ⏭️
shutdown_reply shutdown_request -
status_busy_idle_lifecycle status -
stdin_input_request input_request ⏭️ ⏭️ ⏭️ ⏭️ - ⏭️
update_display_data update_display_data ⚠️ ⚠️ ⏭️ ⚠️ ⏭️ ⏭️ ⏭️ ⏭️ - ⚠️ ⏭️ ⏭️

@github-actions
Copy link
Copy Markdown

Kernel Conformance Matrix

Kernel Protocol Tier 1 Tier 2 Tier 3 Tier 4 Total
almond 5.4 6/11 4/7 1/4 1/5 12/27
ark 5.4 11/11 7/7 3/4 5/5 26/27
async-kernel 5.5 11/11 7/7 4/4 5/5 27/27
deno 5.3 10/11 7/7 4/4 5/5 26/27
evcxr 5.3 10/11 3/7 2/4 3/5 18/27
gonb 5.4 6/11 1/7 1/4 3/5 11/27
ijulia 5.4 10/11 6/7 3/4 5/5 24/27
ipykernel 5.3 11/11 7/7 4/4 5/5 27/27
ocaml-jupyter 5.2 9/11 4/7 2/4 1/5 16/27
xeus-cling 5.3 10/11 4/7 0/4 0/5 14/27
xeus-cpp 5.6 5/11 0/7 0/4 0/5 5/27
xeus-haskell 5.6 7/11 5/7 1/4 2/5 15/27
xeus-lua 5.6 10/11 6/7 1/4 5/5 22/27
xeus-python 5.6 11/11 7/7 4/4 5/5 27/27
xeus-r 5.6 11/11 7/7 3/4 5/5 26/27
xeus-sql 5.6 8/11 5/7 0/4 4/5 17/27
xeus-sqlite 5.6 8/11 6/7 0/4 4/5 18/27

almond (scala) - 15 failures, 0 skipped

Language: scala | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_reply_ok execute_request harness_error Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
status_busy_idle_lifecycle status unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
execute_input_broadcast execute_input unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field history
error_handling execute_request unknown Protocol error: Error deserializing content for msg_type error: missing field traceback
execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mid\u{1b}[39m: \u{1b}[32mString\u{1b}[39m = \u{1b}[32m"586e0482-f019-4ba9-8bc1-813561241151"\u{1b}[39m", expected internally tagged enum ExpressionResult
rich_execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mres11\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m2\u{1b}[39m", expected internally tagged enum ExpressionResult
stdin_input_request input_request harness_error Timeout waiting for iopub idle (stdin test)
comms_lifecycle comm_open harness_error Kernel unresponsive after comm: Timeout waiting for iopub idle
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status
execution_count_increments execute_request unknown Counts didn't increment: 12 -> 0
shutdown_reply shutdown_request unknown Protocol error: Error deserializing content for msg_type shutdown_reply: missing field status
ark () - 1 failures, 0 skipped

Language: R | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
deno (Deno kernel) - 1 failures, 0 skipped

Language: typescript | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
evcxr (evcxr_jupyter) - 6 failures, 3 skipped

Language: Rust | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • stdin_input_request: Kernel can request input from frontend via stdin channel
gonb (gonb) - 15 failures, 1 skipped

Language: go | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
complete_request complete_request unknown Timeout waiting for shell reply
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
is_complete_incomplete is_complete_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
ijulia (ijulia) - 2 failures, 1 skipped

Language: julia | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
comm_info_request comm_info_request unknown Timeout waiting for shell reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
ocaml-jupyter (ocaml-jupyter) - 9 failures, 2 skipped

Language: OCaml | Protocol: 5.2

Failures

Test Message Type Likely Source Reason
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field status
comm_info_request comm_info_request unknown Timeout waiting for shell reply
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Timeout waiting for iopub idle
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-cling (xeus-cling) - 11 failures, 2 skipped

Language: c++ | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown inspect_reply status: Error
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: data did not match any variant of untagged enum HistoryEntry
comm_info_request comm_info_request unknown Timeout waiting for shell reply
display_data display_data unknown Timeout waiting for iopub idle
update_display_data update_display_data unknown Timeout waiting for iopub idle
execute_result execute_result unknown Protocol error: Codec Error: Broken pipe (os error 32)
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-cpp (xeus-cpp) - 20 failures, 2 skipped

Language: C++ | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
execute_stderr stream unknown Timeout waiting for iopub idle
execute_reply_ok execute_request harness_error Protocol error: Codec Error: Broken pipe (os error 32)
status_busy_idle_lifecycle status unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_input_broadcast execute_input unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
complete_request complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
inspect_request inspect_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_complete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_incomplete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
history_request history_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comm_info_request comm_info_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
error_handling execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
display_data display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
update_display_data update_display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_result execute_result unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-haskell (xhaskell) - 8 failures, 4 skipped

Language: haskell | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Protocol error: Codec Error: Broken pipe (os error 32)
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-lua (xlua) - 3 failures, 2 skipped

Language: lua | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stderr stream unknown No stderr containing 'error'
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-r (xr) - 1 failures, 0 skipped

Language: R | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
xeus-sql (xsql) - 7 failures, 3 skipped

Language: MySQL | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-sqlite (xsqlite) - 5 failures, 4 skipped

Language: sql | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel

Test Results by Message Type

Test Message Type almond ark async-kernel deno evcxr gonb ijulia ipykernel ocaml-jupyter xeus-cling xeus-cpp xeus-haskell xeus-lua xeus-python xeus-r xeus-sql xeus-sqlite
comm_info_request comm_info_request
comms_lifecycle comm_open
complete_request complete_request
display_data display_data ⏭️ ⏭️ ⏭️ ⏭️
error_handling execute_request
execute_input_broadcast execute_input
execute_reply_ok execute_request
execute_result execute_result
execute_stderr stream
execute_stdout execute_request
execution_count_increments execute_request
heartbeat_responds heartbeat
history_request history_request
inspect_request inspect_request
interrupt_request interrupt_request
iopub_welcome iopub_welcome ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
is_complete_complete is_complete_request ⚠️ ⚠️
is_complete_incomplete is_complete_request ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
kernel_info_has_language_info kernel_info_request
kernel_info_has_protocol_version kernel_info_request
kernel_info_reply_valid kernel_info_request
parent_header_correlation parent_header
rich_execute_result execute_result ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
shutdown_reply shutdown_request
status_busy_idle_lifecycle status
stdin_input_request input_request ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
update_display_data update_display_data ⚠️ ⚠️ ⏭️ ⚠️ ⏭️ ⏭️ ⏭️ ⏭️ ⚠️ ⏭️ ⏭️

@github-actions
Copy link
Copy Markdown

Kernel Conformance Matrix

Kernel Protocol Tier 1 Tier 2 Tier 3 Tier 4 Total
almond 5.4 6/11 4/7 1/4 1/5 12/27
ark 5.4 11/11 7/7 3/4 5/5 26/27
async-kernel 5.5 11/11 7/7 4/4 5/5 27/27
deno 5.3 10/11 7/7 4/4 5/5 26/27
evcxr 5.3 10/11 3/7 2/4 3/5 18/27
gonb 5.4 6/11 1/7 1/4 3/5 11/27
ijulia 5.4 10/11 6/7 3/4 5/5 24/27
ipykernel 5.3 11/11 7/7 4/4 5/5 27/27
xeus-cling 5.3 10/11 4/7 0/4 0/5 14/27
xeus-cpp 5.6 5/11 0/7 0/4 0/5 5/27
xeus-lua 5.6 10/11 6/7 1/4 5/5 22/27
xeus-octave unknown 0/1 0/0 0/0 0/0 0/1
xeus-python 5.6 11/11 7/7 4/4 5/5 27/27
xeus-r 5.6 11/11 7/7 3/4 5/5 26/27
xeus-sql 5.6 8/11 5/7 0/4 4/5 17/27
xeus-sqlite 5.6 8/11 6/7 0/4 4/5 18/27

almond (scala) - 15 failures, 0 skipped

Language: scala | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_reply_ok execute_request harness_error Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
status_busy_idle_lifecycle status unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
execute_input_broadcast execute_input unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field history
error_handling execute_request unknown Protocol error: Error deserializing content for msg_type error: missing field traceback
execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mid\u{1b}[39m: \u{1b}[32mString\u{1b}[39m = \u{1b}[32m"92ea10c5-2514-4357-bf39-ec7bb08772ef"\u{1b}[39m", expected internally tagged enum ExpressionResult
rich_execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mres11\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m2\u{1b}[39m", expected internally tagged enum ExpressionResult
stdin_input_request input_request harness_error Timeout waiting for iopub idle (stdin test)
comms_lifecycle comm_open harness_error Kernel unresponsive after comm: Timeout waiting for iopub idle
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status
execution_count_increments execute_request unknown Counts didn't increment: 12 -> 0
shutdown_reply shutdown_request unknown Protocol error: Error deserializing content for msg_type shutdown_reply: missing field status
ark () - 1 failures, 0 skipped

Language: R | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
deno (Deno kernel) - 1 failures, 0 skipped

Language: typescript | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
evcxr (evcxr_jupyter) - 6 failures, 3 skipped

Language: Rust | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • stdin_input_request: Kernel can request input from frontend via stdin channel
gonb (gonb) - 15 failures, 1 skipped

Language: go | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
complete_request complete_request unknown Timeout waiting for shell reply
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
is_complete_incomplete is_complete_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
ijulia (ijulia) - 2 failures, 1 skipped

Language: julia | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
comm_info_request comm_info_request unknown Timeout waiting for shell reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
xeus-cling (xeus-cling) - 11 failures, 2 skipped

Language: c++ | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown inspect_reply status: Error
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: data did not match any variant of untagged enum HistoryEntry
comm_info_request comm_info_request unknown Timeout waiting for shell reply
display_data display_data unknown Timeout waiting for iopub idle
update_display_data update_display_data unknown Timeout waiting for iopub idle
execute_result execute_result unknown Protocol error: Codec Error: Broken pipe (os error 32)
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-cpp (xeus-cpp) - 20 failures, 2 skipped

Language: C++ | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
execute_stderr stream unknown Timeout waiting for iopub idle
execute_reply_ok execute_request harness_error Protocol error: Codec Error: Broken pipe (os error 32)
status_busy_idle_lifecycle status unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_input_broadcast execute_input unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
complete_request complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
inspect_request inspect_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_complete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_incomplete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
history_request history_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comm_info_request comm_info_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
error_handling execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
display_data display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
update_display_data update_display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_result execute_result unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-lua (xlua) - 3 failures, 2 skipped

Language: lua | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stderr stream unknown No stderr containing 'error'
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-octave (unknown) - 1 failures, 0 skipped

Language: Octave | Protocol: unknown

Failures

Test Message Type Likely Source Reason
kernel_startup kernel_info_request protocol_error Kernel launch failed: Kernel process exited with signal: 6 (SIGABRT) (core dumped) before connections could be established
xeus-r (xr) - 1 failures, 0 skipped

Language: R | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
xeus-sql (xsql) - 7 failures, 3 skipped

Language: MySQL | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-sqlite (xsqlite) - 5 failures, 4 skipped

Language: sql | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel

Test Results by Message Type

Test Message Type almond ark async-kernel deno evcxr gonb ijulia ipykernel xeus-cling xeus-cpp xeus-lua xeus-octave xeus-python xeus-r xeus-sql xeus-sqlite
comm_info_request comm_info_request -
comms_lifecycle comm_open -
complete_request complete_request -
display_data display_data ⏭️ - ⏭️ ⏭️
error_handling execute_request -
execute_input_broadcast execute_input -
execute_reply_ok execute_request -
execute_result execute_result -
execute_stderr stream -
execute_stdout execute_request -
execution_count_increments execute_request -
heartbeat_responds heartbeat -
history_request history_request -
inspect_request inspect_request -
interrupt_request interrupt_request -
iopub_welcome iopub_welcome ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ -
is_complete_complete is_complete_request - ⚠️
is_complete_incomplete is_complete_request ⚠️ ⚠️ ⚠️ - ⚠️ ⚠️
kernel_info_has_language_info kernel_info_request -
kernel_info_has_protocol_version kernel_info_request -
kernel_info_reply_valid kernel_info_request -
kernel_startup kernel_info_request - - - - - - - - - - - - - - -
parent_header_correlation parent_header -
rich_execute_result execute_result ⏭️ ⏭️ ⏭️ ⏭️ - ⏭️ ⏭️
shutdown_reply shutdown_request -
status_busy_idle_lifecycle status -
stdin_input_request input_request ⏭️ ⏭️ ⏭️ - ⏭️
update_display_data update_display_data ⚠️ ⚠️ ⏭️ ⚠️ ⏭️ ⏭️ - ⚠️ ⏭️ ⏭️

@github-actions
Copy link
Copy Markdown

Kernel Conformance Matrix

Kernel Protocol Tier 1 Tier 2 Tier 3 Tier 4 Total
almond 5.4 6/11 4/7 1/4 1/5 12/27
ark 5.4 11/11 7/7 3/4 5/5 26/27
async-kernel 5.5 11/11 7/7 4/4 5/5 27/27
deno 5.3 10/11 7/7 4/4 5/5 26/27
evcxr 5.3 10/11 3/7 2/4 3/5 18/27
gonb 5.4 6/11 1/7 1/4 3/5 11/27
ijulia 5.4 10/11 6/7 3/4 5/5 24/27
ipykernel 5.3 11/11 7/7 4/4 5/5 27/27
ocaml-jupyter 5.2 9/11 4/7 2/4 1/5 16/27
xeus-cling 5.3 10/11 4/7 0/4 0/5 14/27
xeus-cpp 5.6 5/11 0/7 0/4 0/5 5/27
xeus-haskell 5.6 7/11 5/7 1/4 2/5 15/27
xeus-lua 5.6 10/11 6/7 1/4 5/5 22/27
xeus-octave 5.6 11/11 6/7 0/4 4/5 21/27
xeus-python 5.6 11/11 7/7 4/4 5/5 27/27
xeus-r 5.6 11/11 7/7 3/4 5/5 26/27
xeus-sql 5.6 8/11 5/7 0/4 4/5 17/27
xeus-sqlite 5.6 8/11 6/7 0/4 4/5 18/27

almond (scala) - 15 failures, 0 skipped

Language: scala | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_reply_ok execute_request harness_error Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
status_busy_idle_lifecycle status unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
execute_input_broadcast execute_input unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field history
error_handling execute_request unknown Protocol error: Error deserializing content for msg_type error: missing field traceback
execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mid\u{1b}[39m: \u{1b}[32mString\u{1b}[39m = \u{1b}[32m"8fc2a925-485b-4bfc-bccd-594bd59c0cf7"\u{1b}[39m", expected internally tagged enum ExpressionResult
rich_execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mres11\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m2\u{1b}[39m", expected internally tagged enum ExpressionResult
stdin_input_request input_request harness_error Timeout waiting for iopub idle (stdin test)
comms_lifecycle comm_open harness_error Kernel unresponsive after comm: Timeout waiting for iopub idle
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status
execution_count_increments execute_request unknown Counts didn't increment: 12 -> 0
shutdown_reply shutdown_request unknown Protocol error: Error deserializing content for msg_type shutdown_reply: missing field status
ark () - 1 failures, 0 skipped

Language: R | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
deno (Deno kernel) - 1 failures, 0 skipped

Language: typescript | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
evcxr (evcxr_jupyter) - 6 failures, 3 skipped

Language: Rust | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • stdin_input_request: Kernel can request input from frontend via stdin channel
gonb (gonb) - 15 failures, 1 skipped

Language: go | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
complete_request complete_request unknown Timeout waiting for shell reply
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
is_complete_incomplete is_complete_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
ijulia (ijulia) - 2 failures, 1 skipped

Language: julia | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
comm_info_request comm_info_request unknown Timeout waiting for shell reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
ocaml-jupyter (ocaml-jupyter) - 9 failures, 2 skipped

Language: OCaml | Protocol: 5.2

Failures

Test Message Type Likely Source Reason
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field status
comm_info_request comm_info_request unknown Timeout waiting for shell reply
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Timeout waiting for iopub idle
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-cling (xeus-cling) - 11 failures, 2 skipped

Language: c++ | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown inspect_reply status: Error
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: data did not match any variant of untagged enum HistoryEntry
comm_info_request comm_info_request unknown Timeout waiting for shell reply
display_data display_data unknown Timeout waiting for iopub idle
update_display_data update_display_data unknown Timeout waiting for iopub idle
execute_result execute_result unknown Protocol error: Codec Error: Broken pipe (os error 32)
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-cpp (xeus-cpp) - 20 failures, 2 skipped

Language: C++ | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
execute_stderr stream unknown Timeout waiting for iopub idle
execute_reply_ok execute_request harness_error Protocol error: Codec Error: Broken pipe (os error 32)
status_busy_idle_lifecycle status unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_input_broadcast execute_input unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
complete_request complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
inspect_request inspect_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_complete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_incomplete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
history_request history_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comm_info_request comm_info_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
error_handling execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
display_data display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
update_display_data update_display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_result execute_result unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-haskell (xhaskell) - 8 failures, 4 skipped

Language: haskell | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Protocol error: Codec Error: Broken pipe (os error 32)
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-lua (xlua) - 3 failures, 2 skipped

Language: lua | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stderr stream unknown No stderr containing 'error'
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-octave (xoctave) - 2 failures, 4 skipped

Language: Octave | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-r (xr) - 1 failures, 0 skipped

Language: R | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
xeus-sql (xsql) - 7 failures, 3 skipped

Language: MySQL | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-sqlite (xsqlite) - 5 failures, 4 skipped

Language: sql | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel

Test Results by Message Type

Test Message Type almond ark async-kernel deno evcxr gonb ijulia ipykernel ocaml-jupyter xeus-cling xeus-cpp xeus-haskell xeus-lua xeus-octave xeus-python xeus-r xeus-sql xeus-sqlite
comm_info_request comm_info_request
comms_lifecycle comm_open
complete_request complete_request
display_data display_data ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
error_handling execute_request
execute_input_broadcast execute_input
execute_reply_ok execute_request
execute_result execute_result
execute_stderr stream
execute_stdout execute_request
execution_count_increments execute_request
heartbeat_responds heartbeat
history_request history_request
inspect_request inspect_request
interrupt_request interrupt_request
iopub_welcome iopub_welcome ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
is_complete_complete is_complete_request ⚠️ ⚠️
is_complete_incomplete is_complete_request ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
kernel_info_has_language_info kernel_info_request
kernel_info_has_protocol_version kernel_info_request
kernel_info_reply_valid kernel_info_request
parent_header_correlation parent_header
rich_execute_result execute_result ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
shutdown_reply shutdown_request
status_busy_idle_lifecycle status
stdin_input_request input_request ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
update_display_data update_display_data ⚠️ ⚠️ ⏭️ ⚠️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⚠️ ⏭️ ⏭️

xeus-octave was crashing during startup because Octave could not find
its built-in .m script files (specifically graphics_toolkit). This
happened because the OCTAVE_HOME environment variable was not set when
the test harness spawned the kernel process.

Changes:
- Set OCTAVE_HOME and OCTAVE_EXEC_PREFIX env vars in CI for xeus-octave
- Increase per-test timeout to 30s for xeus-octave (Octave is slow to init)
- Detect kernel process crash before attempting ZMQ connections
- Capture kernel stderr for better diagnostics on startup failures
- Retry kernel_info_request up to 3 times for slow-starting kernels
- Increase initial kernel startup delay from 500ms to 2s
@rgbkrk rgbkrk force-pushed the quill/add-xeus-octave branch from 1c41b60 to 2719876 Compare March 31, 2026 16:49
@github-actions
Copy link
Copy Markdown

Kernel Conformance Matrix

Kernel Protocol Tier 1 Tier 2 Tier 3 Tier 4 Total
almond 5.4 6/11 4/7 1/4 1/5 12/27
ark 5.4 11/11 7/7 3/4 5/5 26/27
async-kernel 5.5 11/11 7/7 4/4 5/5 27/27
deno 5.3 10/11 7/7 4/4 5/5 26/27
evcxr 5.3 10/11 3/7 2/4 3/5 18/27
gonb 5.4 6/11 1/7 1/4 3/5 11/27
ijulia 5.4 10/11 6/7 3/4 5/5 24/27
ipykernel 5.3 11/11 7/7 4/4 5/5 27/27
ocaml-jupyter 5.2 9/11 4/7 2/4 1/5 16/27
xeus-cling 5.3 9/11 4/7 0/4 0/5 13/27
xeus-cpp 5.6 5/11 0/7 0/4 0/5 5/27
xeus-haskell 5.6 7/11 5/7 1/4 2/5 15/27
xeus-lua 5.6 10/11 6/7 1/4 5/5 22/27
xeus-octave 5.6 10/11 1/7 0/4 4/5 15/27
xeus-python 5.6 11/11 7/7 4/4 5/5 27/27
xeus-r 5.6 11/11 7/7 3/4 5/5 26/27
xeus-sql 5.6 8/11 5/7 0/4 4/5 17/27
xeus-sqlite 5.6 8/11 6/7 0/4 4/5 18/27

almond (scala) - 15 failures, 0 skipped

Language: scala | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_reply_ok execute_request harness_error Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
status_busy_idle_lifecycle status unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
execute_input_broadcast execute_input unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mx\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m1\u{1b}[39m", expected internally tagged enum ExpressionResult
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field history
error_handling execute_request unknown Protocol error: Error deserializing content for msg_type error: missing field traceback
execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mid\u{1b}[39m: \u{1b}[32mString\u{1b}[39m = \u{1b}[32m"ddb2029b-8881-4287-8cb9-e1ee6dc638a7"\u{1b}[39m", expected internally tagged enum ExpressionResult
rich_execute_result execute_result unknown Protocol error: Error deserializing content for msg_type execute_reply: invalid type: string "\u{1b}[36mres11\u{1b}[39m: \u{1b}[32mInt\u{1b}[39m = \u{1b}[32m2\u{1b}[39m", expected internally tagged enum ExpressionResult
stdin_input_request input_request harness_error Timeout waiting for iopub idle (stdin test)
comms_lifecycle comm_open harness_error Kernel unresponsive after comm: Timeout waiting for iopub idle
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status
execution_count_increments execute_request unknown Counts didn't increment: 12 -> 0
shutdown_reply shutdown_request unknown Protocol error: Error deserializing content for msg_type shutdown_reply: missing field status
ark () - 1 failures, 0 skipped

Language: R | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
deno (Deno kernel) - 1 failures, 0 skipped

Language: typescript | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
evcxr (evcxr_jupyter) - 6 failures, 3 skipped

Language: Rust | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
interrupt_request interrupt_request unknown Protocol error: Error deserializing content for msg_type interrupt_reply: missing field status

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • stdin_input_request: Kernel can request input from frontend via stdin channel
gonb (gonb) - 15 failures, 1 skipped

Language: go | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
complete_request complete_request unknown Timeout waiting for shell reply
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
is_complete_incomplete is_complete_request unknown Timeout waiting for shell reply
history_request history_request unknown Timeout waiting for shell reply
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
ijulia (ijulia) - 2 failures, 1 skipped

Language: julia | Protocol: 5.4

Failures

Test Message Type Likely Source Reason
comm_info_request comm_info_request unknown Timeout waiting for shell reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
ocaml-jupyter (ocaml-jupyter) - 9 failures, 2 skipped

Language: OCaml | Protocol: 5.2

Failures

Test Message Type Likely Source Reason
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: missing field status
comm_info_request comm_info_request unknown Timeout waiting for shell reply
stdin_input_request input_request unexpected_content No input_request received on stdin channel
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Timeout waiting for iopub idle
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-cling (xeus-cling) - 12 failures, 2 skipped

Language: c++ | Protocol: 5.3

Failures

Test Message Type Likely Source Reason
inspect_request inspect_request unknown inspect_reply status: Error
history_request history_request unknown Protocol error: Error deserializing content for msg_type history_reply: data did not match any variant of untagged enum HistoryEntry
comm_info_request comm_info_request unknown Timeout waiting for shell reply
display_data display_data unknown Timeout waiting for iopub idle
update_display_data update_display_data unknown Timeout waiting for iopub idle
execute_result execute_result unknown Protocol error: Codec Error: Broken pipe (os error 32)
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-cpp (xeus-cpp) - 20 failures, 2 skipped

Language: C++ | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
execute_stderr stream unknown Timeout waiting for iopub idle
execute_reply_ok execute_request harness_error Protocol error: Codec Error: Broken pipe (os error 32)
status_busy_idle_lifecycle status unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_input_broadcast execute_input unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
complete_request complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
inspect_request inspect_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_complete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
is_complete_incomplete is_complete_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
history_request history_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comm_info_request comm_info_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
error_handling execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
display_data display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
update_display_data update_display_data unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
execute_result execute_result unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
comms_lifecycle comm_open harness_error Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
interrupt_request interrupt_request unknown Timeout waiting for control reply
execution_count_increments execute_request unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
parent_header_correlation parent_header unknown Protocol error: Failed to deliver message (6 frames) cause of Not connected to peers. Unable to send messages
shutdown_reply shutdown_request unknown Protocol error: Codec Error: Broken pipe (os error 32)

Skipped (Not Implemented)

  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-haskell (xhaskell) - 8 failures, 4 skipped

Language: haskell | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
comm_info_request comm_info_request unknown Timeout waiting for shell reply
execution_count_increments execute_request unknown Timeout waiting for iopub idle
parent_header_correlation parent_header unknown Protocol error: Codec Error: Broken pipe (os error 32)
shutdown_reply shutdown_request unknown Timeout waiting for control reply

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-lua (xlua) - 3 failures, 2 skipped

Language: lua | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stderr stream unknown No stderr containing 'error'
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-octave (xoctave) - 8 failures, 4 skipped

Language: Octave | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown Timeout waiting for iopub idle
complete_request complete_request unknown Expected complete_reply, got "execute_reply"
inspect_request inspect_request unknown Expected inspect_reply, got "execute_reply"
is_complete_complete is_complete_request unknown Expected is_complete_reply, got "inspect_reply"
history_request history_request unknown Expected history_reply, got "execute_reply"
comm_info_request comm_info_request unknown Expected comm_info_reply, got "history_reply"
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel
xeus-r (xr) - 1 failures, 0 skipped

Language: R | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
xeus-sql (xsql) - 7 failures, 3 skipped

Language: MySQL | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub
stdin_input_request input_request unexpected_content No input_request received on stdin channel

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
xeus-sqlite (xsqlite) - 5 failures, 4 skipped

Language: sql | Protocol: 5.6

Failures

Test Message Type Likely Source Reason
execute_stdout execute_request unknown No stdout containing 'hello'
execute_stderr stream unknown No stderr containing 'error'
execute_reply_ok execute_request kernel_error execute_reply status: Error
execute_result execute_result unknown No execute_result on iopub

Skipped (Not Implemented)

  • display_data: Kernel can produce display_data messages for rich output
  • update_display_data: Kernel can update existing displays via update_display_data
  • rich_execute_result: Expression evaluation produces execute_result with rich MIME types (HTML, images, etc.)
  • stdin_input_request: Kernel can request input from frontend via stdin channel

Test Results by Message Type

Test Message Type almond ark async-kernel deno evcxr gonb ijulia ipykernel ocaml-jupyter xeus-cling xeus-cpp xeus-haskell xeus-lua xeus-octave xeus-python xeus-r xeus-sql xeus-sqlite
comm_info_request comm_info_request
comms_lifecycle comm_open
complete_request complete_request
display_data display_data ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
error_handling execute_request
execute_input_broadcast execute_input
execute_reply_ok execute_request
execute_result execute_result
execute_stderr stream
execute_stdout execute_request
execution_count_increments execute_request
heartbeat_responds heartbeat
history_request history_request
inspect_request inspect_request
interrupt_request interrupt_request
iopub_welcome iopub_welcome ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
is_complete_complete is_complete_request ⚠️ ⚠️
is_complete_incomplete is_complete_request ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️
kernel_info_has_language_info kernel_info_request
kernel_info_has_protocol_version kernel_info_request
kernel_info_reply_valid kernel_info_request
parent_header_correlation parent_header
rich_execute_result execute_result ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
shutdown_reply shutdown_request
status_busy_idle_lifecycle status
stdin_input_request input_request ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️
update_display_data update_display_data ⚠️ ⚠️ ⏭️ ⚠️ ⏭️ ⏭️ ⏭️ ⏭️ ⏭️ ⚠️ ⏭️ ⏭️

@rgbkrk rgbkrk marked this pull request as ready for review March 31, 2026 19:07
@rgbkrk rgbkrk merged commit b3c08be into main Mar 31, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant