Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ target/

# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
Thumbs.db
46 changes: 30 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,16 @@ to solving common JNI use-cases and then present the results.
The benchmarks at present are:

1. [com.evolvedbinary.jnibench.common.call](tree/main/src/main/java/com/evolvedbinary/jnibench/common/call) -
Benchmarks for [Creating Objects with JNI](ObjectCreationBenchmarks.md)
[(results)](ObjectCreationBenchmarks.md#object-creation-results).
Benchmarks for [Creating Objects with JNI](ObjectCreationBenchmarks.md)
[(results)](ObjectCreationBenchmarks.md#object-creation-results).

2. [com.evolvedbinary.jnibench.common.array](tree/main/src/main/java/com/evolvedbinary/jnibench/common/array) -
Benchmarks for [Passing Arrays with JNI](ArrayPassingBenchmarks.md)
[(results)](ArrayPassingBenchmarks.md#array-passing-results).
Benchmarks for [Passing Arrays with JNI](ArrayPassingBenchmarks.md)
[(results)](ArrayPassingBenchmarks.md#array-passing-results).

3. [com.evolvedbinary.jnibench.common.bytearray](tree/main/src/main/java/com/evolvedbinary/jnibench/common/bytearray), and [com.evolvedbinary.jnibench.common.getputjni](tree/main/src/main/java/com/evolvedbinary/jnibench/common/getputjni) - Benchmarks for [JNI Data Transfer](DataBenchmarks.md).
3. [com.evolvedbinary.jnibench.common.bytearray](tree/main/src/main/java/com/evolvedbinary/jnibench/common/bytearray),
and [com.evolvedbinary.jnibench.common.getputjni](tree/main/src/main/java/com/evolvedbinary/jnibench/common/getputjni) -
Benchmarks for [JNI Data Transfer](DataBenchmarks.md).

# Reproducing

Expand All @@ -40,29 +42,41 @@ can use `benchmark-100.sh` and/or `benchmark-100-with-close.sh`, or

## JMH support

We have support for running the tests via JMH, see `jmh-benchmarks.sh`. You can
also pass `--help` to the script to see JMH options.

### Byte array benchmarks

There are two benchmarks, which are currently available only via JMH:
ByteArrayFromNativeBenchmark and ByteArrayToNativeBenchmark. They can be run
multiple times using `jmh-benchmarks-parametrized.sh` with:
multiple times using `jmhrun.py` with:

```bash
python3 jmhrun.py -c jmh_tiny_array_from_native.json
```

The above command will run JMH with `ByteArrayFromNativeBenchmark` and store the result in CSV files in a subdirectory
under `results` named via timestamp. You can also pass `--help` to the script to see additional JMH options that can be
used.

Results can then be plotted using the `jmhplot.py` script. For example, to produce results
for the `ByteArrayFromNativeBenchmark` example above, you can run:

```bash
./jmh-benchmarks-parametrized.sh -i 10 -b ByteArrayToNativeBenchmark -o results/ -f csv
python3 jmhplot.py -f results/jmh_2026-01-29T12:11:26.472664
```

The above command will run JMH with `ByteArrayToNativeBenchmark` benchmarks `10` times and store result in CSV files in `results` directory. You can also pass `--help` to the script to see additional JMH options that can be used.
(Replace the `-f` parameter with the appropriate directory path.)

Command line parameter `f` expects a path to the directory with the JMH result CSV files from running the benchmarks
with `jmhrun.py`.
Some benchmarks will require an additional parameter `-c` like this:

Results can then be plotted using the `process_byte_array_benchmarks_results.py` script. For example, to produce results for the `ByteArrayToNativeBenchmark` benchmarks, you can run:
```bash
python3 process_byte_array_benchmarks_results.py -p results/ --param-name "Param: keySize" --chart-title "Performance comparison of passing byte array with {} bytes via JNI"
python3 jmhplot.py -f results/jmh_2026-01-29T12:11:26.472664 -c jmh_plot_iterator.json
```

Command line parameter `p` expects a path to the directory with the JMH result CSV files from running the benchmarks with `jmh-benchmarks-parametrized.sh`.
The `{}` in the `chart-title` parameter will be replaced by the value from the `param-name` column.
The script creates the resulting graphs in the same directory as the one where it finds the CSV files.

# Other Resources
1. Java Foreign Interface prototype and performance results in RocksJava (i.e. Panama) - https://rocksdb.org/blog/2024/02/20/foreign-function-interface.html

1. Java Foreign Interface prototype and performance results in RocksJava (i.e.
Panama) - https://rocksdb.org/blog/2024/02/20/foreign-function-interface.html
2. RocksJava API Performance Improvements - https://rocksdb.org/blog/2023/11/06/java-jni-benchmarks.html
Binary file added analysis/arrayFromNative/fig_65536_allbig.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added analysis/arrayFromNative/fig_65536_allsmall.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added analysis/arrayFromNative/fig_65536_ba.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added analysis/arrayFromNative/fig_65536_nopoolbig.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 41 additions & 0 deletions analysis/arrayFromNative/jmh_2026-01-16T12:19:55.210351.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
## JMH Run
This JMH run was generated on 2026-01-16T12:19:55.210351
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_run_array_from_native.json`
```json
{
"help": false,
"benchmark": "ByteArrayFromNativeBenchmark",
"jvmargs": [
"Xmx24G",
"XX:+HeapDumpOnOutOfMemoryError"
],
"params": {
"valueSize": [
50,
4096
],
"cacheMB": [
1
],
"checksum": [
"none"
]
},
"options": {
"batchsize": 2,
"warmupiterations": 1,
"warmuptime": "10us",
"iterations": 1,
"time": "10us"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx24G -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmark -p valueSize=50,4096 -p cacheMB=1 -p checksum=none -bs 2 -wi 1 -w 10us -i 1 -r 10us -rff results/jmh_2026-01-16T12:19:55.210351/jmh_2026-01-16T12:19:55.210351.csv
```
3 changes: 3 additions & 0 deletions analysis/arrayFromNative/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"1.8.0_472\""
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
## JMH Run
This JMH run was generated on 2026-01-19T16:05:42.022395
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_tiny_array_from_native_java21.json`
```json
{
"benchmark": "ByteArrayFromNativeBenchmarkJava21",
"jvmargs": [
"Xmx4G",
"XX:ErrorFile=./results/hs_err_pid%p.log",
"XX:+HeapDumpOnOutOfMemoryError",
"-enable-preview"
],
"params": {
"valueSize": [
50,
1024,
4096,
16384
],
"cacheMB": [
1
],
"checksum": [
"none",
"copyout"
]
},
"options": {
"batchsize": 1,
"warmupiterations": 5,
"warmuptime": "10ms",
"iterations": 5,
"time": "50ms"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx4G -XX:ErrorFile=./results/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError --enable-preview -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmarkJava21 -p valueSize=50,1024,4096,16384 -p cacheMB=1 -p checksum=none,copyout -bs 1 -wi 5 -w 10ms -i 5 -r 50ms -rff results/jmh_2026-01-19T16:05:42.022395/jmh_2026-01-19T16:05:42.022395.csv
```
3 changes: 3 additions & 0 deletions analysis/byteArrayFromNativeJava21/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"21.0.9\" 2025-10-21"
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
## JMH Run
This JMH run was generated on 2026-01-19T15:41:17.308765
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_tiny_array_from_native_java25.json`
```json
{
"benchmark": "ByteArrayFromNativeBenchmarkJava25",
"jvmargs": [
"Xmx4G",
"XX:ErrorFile=./results/hs_err_pid%p.log",
"XX:+HeapDumpOnOutOfMemoryError"
],
"params": {
"valueSize": [
50,
1024,
4096,
16384
],
"cacheMB": [
1
],
"checksum": [
"none",
"copyout"
]
},
"options": {
"batchsize": 1,
"warmupiterations": 5,
"warmuptime": "10ms",
"iterations": 5,
"time": "50ms"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx4G -XX:ErrorFile=./results/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmarkJava25 -p valueSize=50,1024,4096,16384 -p cacheMB=1 -p checksum=none,copyout -bs 1 -wi 5 -w 10ms -i 5 -r 50ms -rff results/jmh_2026-01-19T15:41:17.308765/jmh_2026-01-19T15:41:17.308765.csv
```
3 changes: 3 additions & 0 deletions analysis/byteArrayFromNativeJava25/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"25.0.1\" 2025-10-21"
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
## JMH Run
This JMH run was generated on 2026-01-17T09:45:55.943339
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_run_array_from_native.json`
```json
{
"help": false,
"benchmark": "ByteArrayFromNativeBenchmark",
"jvmargs": [
"Xmx24G",
"XX:+HeapDumpOnOutOfMemoryError"
],
"params": {
"valueSize": [
50,
4096
],
"cacheMB": [
1
],
"checksum": [
"none"
]
},
"options": {
"batchsize": 2,
"warmupiterations": 1,
"warmuptime": "10us",
"iterations": 1,
"time": "10us"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx24G -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmark -p valueSize=50,4096 -p cacheMB=1 -p checksum=none -bs 2 -wi 1 -w 10us -i 1 -r 10us -rff results/jmh_2026-01-17T09:45:55.943339/jmh_2026-01-17T09:45:55.943339.csv
```
3 changes: 3 additions & 0 deletions analysis/byteArrayFromNativeJava8/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"1.8.0_472\""
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
## JMH Run
This JMH run was generated on 2026-01-17T10:13:15.573031
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_run_array_from_native.json`
```json
{
"help": false,
"benchmark": "ByteArrayFromNativeBenchmark",
"jvmargs": [
"Xmx24G",
"XX:+HeapDumpOnOutOfMemoryError"
],
"params": {
"valueSize": [
50,
4096
],
"cacheMB": [
1
],
"checksum": [
"none"
]
},
"options": {
"batchsize": 2,
"warmupiterations": 1,
"warmuptime": "10us",
"iterations": 1,
"time": "10us"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx24G -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmark -p valueSize=50,4096 -p cacheMB=1 -p checksum=none -bs 2 -wi 1 -w 10us -i 1 -r 10us -rff results/jmh_2026-01-17T10:13:15.573031/jmh_2026-01-17T10:13:15.573031.csv
```
3 changes: 3 additions & 0 deletions analysis/byteArrayFromNativeJava8_2/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"1.8.0_472\""
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
## JMH Run
This JMH run was generated on 2026-01-17T22:30:09.141651
#### Config
The configuration was read from `/home/rklein/jni-benchmarks/jmh_full_array_from_native.json`
```json
{
"benchmark": "ByteArrayFromNativeBenchmark",
"jvmargs": [
"Xmx4G",
"XX:ErrorFile=./results/hs_err_pid%p.log",
"XX:+HeapDumpOnOutOfMemoryError"
],
"params": {
"valueSize": [
10,
50,
512,
1024,
4096,
8192,
16384,
32768,
65536
],
"cacheMB": [
1
],
"checksum": [
"none",
"copyout"
]
},
"options": {
"batchsize": 1,
"warmupiterations": 20,
"warmuptime": "50ms",
"iterations": 50,
"time": "500ms"
},
"result.path": "./results",
"java.library.path": "target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib",
"jar": "target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar"
}
```
#### Command
The java command executed to run the tests
```
java -Xmx4G -XX:ErrorFile=./results/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=target/jni-benchmarks-1.0.1-SNAPSHOT-application/jni-benchmarks-1.0.1-SNAPSHOT/lib -jar target/jni-benchmarks-1.0.1-SNAPSHOT-benchmarks.nar ByteArrayFromNativeBenchmark -p valueSize=10,50,512,1024,4096,8192,16384,32768,65536 -p cacheMB=1 -p checksum=none,copyout -bs 1 -wi 20 -w 50ms -i 50 -r 500ms -rff results/jmh_2026-01-17T22:30:09.141651/jmh_2026-01-17T22:30:09.141651.csv
```
3 changes: 3 additions & 0 deletions analysis/byteArrayFromNative_full/system_info.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"system_info": "x86_64 - Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz - 125GB RAM - Ubuntu 24.04.3 LTS - Kernel: 6.14.0-36-generic - openjdk version \"1.8.0_472\""
}
Loading