XLS backend

[vasdommes]

Description

This PR adds XLS backend. It is based on PR #1343, with most of the code rewritten and new features added.

Google XLS is an open-source (Apache 2) High Level Syntesis toolchain that produces an RTL (Verilog or SystemVerilog) design from a high-level description (DSLX or C++).

Adding XLS as a new hls4ml backend allows to generate RTL without vendor-specific dependencies and benefit from the developments that XLS brings to HLS field.

XLS workflow

XLS backend performs the following transformations:

• write(): hls4ml representation -> DSLX project
• compile(): DSLX -> XLS IR -> Optimized XLS IR
• build(): Optimized XLS IR -> (System)Verilog -> IP

DSLX -> IR -> (System)Verilog conversion is done by XLS.
IP is generated by Vivado. One can choose another vendor and generate IP from Verilog file manually.

XLS features

XLS backend supports the following layers: Input, ApplyAlpha, BatchNormalization, Dense, Conv1D, DepthwiseConv1D, Conv2D, DepthwiseConv2D, Pooling1D, Pooling2D, GlobalPooling1D, GlobalPooling2D, Merge, Concatenate, Dot, Activation, HardActivation, ParametrizedActivation, PReLU, Reshape, Softmax, Transpose, TernaryTanh.

You can override default codegen options as follows:

config = hls4ml.utils.config_from_keras_model(model)
# This sets hls_model.config['XLSCodegenFlags']
hls_model = hls4ml.converters.convert_from_keras_model(
    model, hls_config=config, backend='XLS',
    xls_codegen_flags={'delay_model': 'asap7', 'generator': 'pipeline', 'use_system_verilog': False}
)

DSLX standard library has only signed FixedPoint type (similar to ap_fixed). Thus, unsigned types are not supported.

Currently, XLS backend implements only IOType: io_parallel. Strategy is ignored.
All operations are fully unrolled.

io_stream could be implemented via DSLX procs. @calad0i and I are going to work on that after finishing this PR.

Other changes

I made some minor changes in non-XLS code:

• Fixed test_softmax.py does not test argmax and latency implementations; latency fails #1443, since it was needed to test all softmax implementations in XLS.
• Added hook to ModelGraph to call custom backend.get_top_function(). This is needed for XLS because it uses optimized XLS IR file instead of .so library generated by other backends.
• Updated docs/ir/attributes.rst. Aside from adding XLS, this commit some other missing layers and attributes.

Dependencies

XLS backend uses xls-python to access XLS API. It is enabled by dependency group xls:

pip install hls4ml[xls]

xls-python comes with batteries (libxls.so and DSLX standard library) included, no separate XLS installation is required.
The code has been tested for the version xls-python=0.1.9875.

Known issues

XLS doesn't work with Dense layer imported from PyTorch Linear layer because of shape mismatch: PyTorch stores Linear weights as (out_features, in_features), while hls4ml Dense layers use the Keras-style layout (in_features, out_features).

Repro: add XLS backend to test_pytorch_api.py/test_squeeze and run the test.

Note that the weights in this test are constant, and other backends flatten them without checking shape.
So, it is unclear whether they handle this situation correctly or not.

Type of change

• Documentation update
• New feature (non-breaking change which adds functionality)

Tests

📝 Please describe the tests that you ran to verify your changes.

• Provide instructions so we can reproduce.
• Please also list any relevant details for your test configuration.

XLS has been added to the following tests:
test_activations.py, test_auto_precision.py, test_binary_cnn.py, test_causalpadding.py, test_depthconv1d.py, test_depthconv2d.py, test_keras_api.py, test_keras_v3_api.py, test_merge.py, test_multi_dense.py, test_pointwiseconv.py, test_pooling.py, test_pytorch_api.py, test_reshape.py, test_sepconv1d.py, test_sepconv2d.py, test_softmax.py.

Test Configuration

Add xls dependency, e.g.

pip install .[da,testing,testing-keras2,sr,optimization,xls]"
# or: pip install .[da,testing,testing-keras3,sr,xls]"

and run tests, e.g.:

pytest test/pytest --randomly-dont-reset-seed -k XLS

Notes on performance

Some test cases are very slow for XLS (e.g. ~30 minutes vs ~10 seconds on other backends).
This happens because XLS generates (in model.compile()) and uses (in model.predict()) an optimized XLS IR code, where all loops are fully unrolled. The resulting file can be huge and thus slow for the likes of Conv2D.

During development, I made test faster by reducing dimensions in some tests.
For example, in test_keras_api.py/test_conv2d I replaced

input_shape = (28, 28, 3)
filters=32

with

input_shape = (14, 14, 3)
filters=8

I haven't pushed such changes, but that could be one of the ways of speeding things up.

UPD: I reduced XLS dimensions in many tests, see 8578a62 and 75a04fb.

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

[vasdommes]

I reduced XLS dimensions in many tests to make them fast enough, but the test https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75768174 failed due to 2hr timeout.

$ pytest $PYTESTFILE -rA --cov-report xml --cov-report term --cov=hls4ml --junitxml=report.xml --randomly-seed=42 --randomly-dont-reorganize --randomly-dont-reset-seed
============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0
pytest-randomly: reseed with 42
Using --randomly-seed=42
rootdir: /builds/fastmachinelearning/hls4ml
configfile: pyproject.toml
plugins: anyio-4.12.0, cov-7.0.0, randomly-4.0.1
collected 227 items / 1 skipped
test_einsum_dense.py ..................                                  [  7%]
test_keras_v3_api.py ...........s....................................... [ 30%]
.Terminated
WARNING: step_script could not run to completion because the timeout was exceeded. For more control over job and script timeouts see: https://docs.gitlab.com/ee/ci/runners/configure_runners.html#set-script-and-after_script-timeouts
ERROR: Job failed: execution took longer than 2h0m0s seconds

That's surprising - on my laptop, test_einsum_dense finishes in 3 min and test_keras_v3_api - in 16 min (including 4 min for XLS).

[calad0i]

This timeout looks like CI issue, not something real in the code.

[vloncar]

If it is not doing synthesis (and it seems that it skips that path), what does it do that takes a long time?

[calad0i]

If it is not doing synthesis (and it seems that it skips that path), what does it do that takes a long time?

I feel like that the slowness is not due to the tests themselves, more likely the runner itself stalls on random tests one in a while...

https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75680057
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75627913
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75482738
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75329237
https://gitlab.cern.ch/fastmachinelearning/hls4ml/-/jobs/75320437

[vloncar]

Hmm, seems that all tests always run only on -05, other nodes are never used. Gitlab reports them as "stale". Turns out they are not configured right, always reporting as -05, but only the real -05 is used. I've reconfigured 06 and 08 to be separate, they are in the pool now. Sadly 07 doesn't respond, will need to ping an openlab admin to configure that one. Hopefully then it will be a bit less crowded and tests will run faster