Nimbus Integration: TCP Backend and Interface Introspection #740

cmoscy · 2025-11-04T08:33:28Z

Added TCP I/O protocol with the Hamilton specific TCPBackend supporting Nimbus (and possibly Prep) specific communication and introspection through their IP->Harp2->Hoi2 protocol.
Currently includes basics of instrument connection, introspection.
Can use HamiltonCommand to construct message to run instrument commands while managing any responses received from the instrument.

…ugh Hamilton .NET firmware interface. TCP connection works through proxy to log direct communication.

- Added tcp_comlink_proxy for setup of Nimbus Comlink instance. Manages TCP communication through .NET firmware libraries - Removed setup.py in favor of pyproject.toml for package management. - Updated pyproject.toml with project metadata, dependencies, and optional dependencies.

- Updated `README.md` to include instructions for using the test notebook and requirements for DLL dependencies. - Introduced `dll_comlink_test.ipynb` to demonstrate TCP communication with Hamilton Nimbus instruments, including connection setup, module discovery, and pipettor operations. - Added `nimbus-connect-validation.json` as logging and validating TCP communication example - Fixed `firmware_assemblies.py` to load required Hamilton.Components.TransportLayer.Protocols.dll

…port - Introduced `tcp_codec.py` for Hamilton protocol communication, implementing a three-layer packet structure (IpPacket, Harp2, Hoi2) and associated message builders for command execution.

- Introduced a new Hamilton TCP backend with connection management, message routing, and command execution capabilities. - Implemented a layered architecture for the Hamilton protocol, including packet structures (IpPacket, HarpPacket, HoiPacket) and message builders (CommandMessage, InitMessage). - HoiParams for automatic DataFragment wrapping in command parameters.

- Added HamiltonIntrospection class for dynamic discovery of instrument capabilities (Thanks Piglet for showing that pattern). - Implemented commands for retrieving object interfaces, metadata, method signatures, enums, and structs. - Example in nimbus_connection_test.ipynb demonstrates finding an interface (Doorlock in this case), and then constructing the corresponding commands to check status, lock, unlock, etc.

1. Improved HoiParameter handling for more efficient use 2. Updated nimbus_connection_test.ipynb to show pipettor introspection, and initialization + park execution 3. Restored original project setup.py and pyproject files for consistency. 4. Moved all DLL related features to separate branch

cmoscy · 2025-11-05T02:15:42Z

My fault with the typing/linting. Cleaning this up and will update shortly

rickwierenga · 2025-11-05T02:17:13Z

it's fine to do those things last

for linting make format might do a lot easily

but otherwise don't bother making tests pass every revision, it's more about content than style at this point

rickwierenga · 2025-11-05T02:18:11Z

do you think it would be possible to load some commands like aspirate and dispense and implement them in python in PLR?

cmoscy · 2025-11-05T02:30:27Z

Got it! Was pretty minor to get it to pass.

That's definitely possible! Primary bottleneck for me has been handling that Nimbus has several deck/waste block configurations. Pretty sure there's a way to detect a lot of this so there's no potentially dangerous assumptions made.

Could add basic instrument control into a NimbusBackend pretty easily in the next 1-2 days (User would have to specify coordinates themselves until we get a deck layout defined).

Tip pickup, drop, aspirate, dispense, initialization, and door control seem like a good minimal implementation? Anything else that would be worth prioritizing?

rickwierenga · 2025-11-05T02:34:05Z

those "big four" + setup are the most important, everything else is nice to have. appreciate it!

for the stars the user has to specify some things about their deck layout, like where the core grippers are if they have them. after that it's just modeled through the universal resource model. we could have a similar thing for the nimbus

deck = NimbusDeck(
  trash_pos = "option a"
)

…ecodes and displays command args and return values into their corresponding types.

cmoscy · 2025-11-05T04:46:31Z

Sounds good, thanks for the example. Checkpointed the cleanup and improved the introspection to help me put all that together next!

…ls for testing. Example notebook with NimbusBackend Examples. ! Corrected message parsing for array types in HoiParamsParser.

- Added NimbusTipType enumeration for mapping tip types to Hamilton protocol commands. - Implemented InitializeSmartRoll, SetChannelConfiguration, PickupTips, DropTips, and DropTipsRoll commands for Nimbus backend. - Updated NimbusBackend setup process to include channel configuration and tip presence checks. - Enhanced NimbusDeck to create default long waste block and associated waste positions. - Improved error handling and logging for tip pickup and drop operations.

cmoscy · 2026-01-19T01:05:44Z

I am surprised PickupTips does not have z_final_positions

Right? PickupGripperTool also lacks a zFinal in the signature. Weird since most other operations have it. But double checked the extracted interface methods and their firmware documentation. They both don't include zFinal.

Pipettor Interface Methods from introspection layer for reference.
pipettor_method_ids.txt

rickwierenga · 2026-01-19T18:17:18Z

pylabrobot/liquid_handling/backends/hamilton/nimbus_backend.py

+    # flow_rate should not be None - if it is, it's an error (no hardcoded fallback)
+    flow_rates: List[float] = []
+    for op in ops:
+      if op.flow_rate is None:
+        raise ValueError(f"flow_rate cannot be None for operation {op}")
+      flow_rates.append(op.flow_rate)
+    blowout_volumes = [
+      op.blow_out_air_volume if op.blow_out_air_volume is not None else 40.0 for op in ops
+    ]  # in uL, default 40


this means the user has to specify flow rates, which isn't the case for other liquid handlers in PLR. they will often want to, but maybe not always (eg for simple prototyping)

what is the default value nimbus uses if anything?

1000 ul tip: 250 asp / 400 disp
300 and 50 ul tip: 100 asp / 180 disp
10 ul tip: 100 asp / 75 disp

rickwierenga · 2026-01-19T22:43:51Z

what is the difference between HoiPacket and HoiResponse?

cmoscy · 2026-01-20T03:23:37Z

what is the difference between HoiPacket and HoiResponse?

HoiPacket is the lower level wire protocol, handles low level encoding/decoding. HoiResponse in the interpreter layer that translates enums, action codes, etc that map to hamilton responses/behaviors.

rickwierenga · 2026-01-20T06:07:10Z

HoiPacket is the lower level wire protocol, handles low level encoding/decoding. HoiResponse in the interpreter layer that translates enums, action codes, etc that map to hamilton responses/behaviors.

It seems they contain the same information?

parse_message checks the action for error, but then in the success case copies information over into a HoiResponse:

 return SuccessResponse(
      action=action,
      interface_id=cmd_response.hoi.interface_id,
      action_id=cmd_response.hoi.action_id,
      raw_params=cmd_response.hoi.params,
      response_required=cmd_response.hoi.response_required,
    )

and it's pretty similar in the error case.

    if isinstance(hoi_response, ErrorResponse):

in send_command could just be if action in (Hoi2Action.STATUS_EXCEPTION, Hoi2Action.COMMAND_EXCEPTION, Hoi2Action.INVALID_ACTION_RESPONSE)

is there a reason I shouldn't merge them into one? (delete HoiResponse probably incl related refactoring)

cmoscy · 2026-01-20T06:13:10Z

HoiPacket is the lower level wire protocol, handles low level encoding/decoding. HoiResponse in the interpreter layer that translates enums, action codes, etc that map to hamilton responses/behaviors.

It seems they contain the same information?

parse_message checks the action for error, but then in the success case copies information over into a HoiResponse:
 return SuccessResponse(
      action=action,
      interface_id=cmd_response.hoi.interface_id,
      action_id=cmd_response.hoi.action_id,
      raw_params=cmd_response.hoi.params,
      response_required=cmd_response.hoi.response_required,
    )
and it's pretty similar in the error case.
    if isinstance(hoi_response, ErrorResponse):
in send_command could just be if action in (Hoi2Action.STATUS_EXCEPTION, Hoi2Action.COMMAND_EXCEPTION, Hoi2Action.INVALID_ACTION_RESPONSE)

is there a reason I shouldn't merge them into one? (delete HoiResponse probably incl related refactoring)

Would be fine with me. I originally kept them separate to stay aligned with Hamilton’s .NET structure — it made it easier to test against the firmware DLLs.

Now that we've captured most of it, probably less important to be a 1:1 mirror.

rickwierenga · 2026-01-20T22:44:40Z

Yes, that's right. Sounds great to make it more consistent across instruments

Unfortunately the spacing between channels is a little odd here:

  Position 1 to 2: 19.863 - 1.880 = 17.983mm                                                                                                                                                                           
  Position 2 to 3: 1.880 - (-76.149) = 78.029mm                                                                                                                                                                        
  Position 3 to 4: -76.149 - (-94.132) = 17.983mm                                                                                                                                                                      
  Position 4 to 5: -94.132 - (-152.349) = 58.217mm                                                                                                                                                                     
  Position 5 to 6: -152.349 - (-170.332) = 17.983mm                                                                                                                                                                    
  Position 6 to 7: -170.332 - (-219.549) = 49.217mm                                                                                                                                                                    
  Position 7 to 8: -219.549 - (-237.532) = 17.983mm

This means we can't use get_tight_single_resource_liquid_op_offsets which is used by lh.discard_tips.

Since I have already decided I want to move having decks define a tip position for every tip spot and having discard use these, it is fine to keep it in this pattern for the nimbus (the new pattern that other backends will also use soon) rather than figuring these things out right now.

rickwierenga · 2026-01-20T22:57:38Z

@cmoscy could you please make sure the code still works as expected? then it will be ready to merge.

rickwierenga · 2026-01-21T01:02:19Z

I am merging the binary parser (https://github.com/PyLabRobot/pylabrobot/pull/740/changes#diff-8e70176e729f055fd4aae4c482c93135752ce382647b1d60f2084a3194c6c902) separately right now so I can use it the tecan infinite PR while finish this up

cmoscy · 2026-01-21T03:55:42Z

@cmoscy could you please make sure the code still works as expected? then it will be ready to merge.

Will take a look tonight

cmoscy · 2026-01-21T03:57:51Z

Yes, that's right. Sounds great to make it more consistent across instruments

Unfortunately the spacing between channels is a little odd here:

  Position 1 to 2: 19.863 - 1.880 = 17.983mm                                                                                                                                                                           
  Position 2 to 3: 1.880 - (-76.149) = 78.029mm                                                                                                                                                                        
  Position 3 to 4: -76.149 - (-94.132) = 17.983mm                                                                                                                                                                      
  Position 4 to 5: -94.132 - (-152.349) = 58.217mm                                                                                                                                                                     
  Position 5 to 6: -152.349 - (-170.332) = 17.983mm                                                                                                                                                                    
  Position 6 to 7: -170.332 - (-219.549) = 49.217mm                                                                                                                                                                    
  Position 7 to 8: -219.549 - (-237.532) = 17.983mm

This means we can't use get_tight_single_resource_liquid_op_offsets which is used by lh.discard_tips.

Since I have already decided I want to move having decks define a tip position for every tip spot and having discard use these, it is fine to keep it in this pattern for the nimbus (the new pattern that other backends will also use soon) rather than figuring these things out right now.

Right on, you reminded me of that little detail on the Nimbus. Funny enough those are the default tip waste positions from their deck definition. Think it avoids the teaching positions in case you don't place the little metal piece that is suposed to cover them when not being actively used on their default long waste.

rickwierenga · 2026-01-21T05:02:44Z

That makes sense.

Also, I made a forum post to discuss making this behavior the default: https://discuss.pylabrobot.org/t/per-channel-tip-discard-positions-on-deck-instead-of-computed-offsets/420 because it is more flexible and generally nicer

…arameters Drop operations now use fixed offsets (10mm) instead of tip length, matching VantageBackend. Fixes regression from 8f2775c.

cmoscy · 2026-01-21T06:50:07Z

Couple regressions, but nothing too bad. I like the consolidation you implemented, will do a little smoothing tomorrow evening. Shouldn't take long.

cmoscy and others added 18 commits August 6, 2025 23:22

Added Nimbus TCP proxy connection and firmware modules. Controls thro…

69305db

…ugh Hamilton .NET firmware interface. TCP connection works through proxy to log direct communication.

Merge remote-tracking branch 'origin/main' into nimbus8

5195996

Merge branch 'PyLabRobot:main' into nimbus8

01bd4d4

Update TCP communication implementation and add Hamilton protocol sup…

9d16f3b

…port - Introduced `tcp_codec.py` for Hamilton protocol communication, implementing a three-layer packet structure (IpPacket, Harp2, Hoi2) and associated message builders for command execution.

Merge branch 'PyLabRobot:main' into nimbus8

e540a1d

Merge branch 'PyLabRobot:main' into nimbus8

7d1623a

Merge branch 'PyLabRobot:main' into nimbus8

28d729f

Cleanup example notebook

0a39c14

ignore

554f3e5

Merge branch 'PyLabRobot:main' into nimbus8

5bcec1a

Remove DLL control features - moved to DLL_CONTROL branch

a385890

Merge branch 'PyLabRobot:main' into nimbus8

da5bcca

Log and ignore cleanup

d4c9067

cmoscy and others added 2 commits November 4, 2025 20:43

Linting/Typing cleanup for PR. Introspection interface now properly d…

c21a937

…ecodes and displays command args and return values into their corresponding types.

Merge branch 'PyLabRobot:main' into nimbus8

32b17fa

cmoscy and others added 4 commits November 4, 2025 20:56

tcp_introspection straggler. Improved return type handling.

221e993

Add Nimbus backend with setup method. Create bare Nimbus deck and uti…

344f213

…ls for testing. Example notebook with NimbusBackend Examples. ! Corrected message parsing for array types in HoiParamsParser.

Merge branch 'PyLabRobot:main' into nimbus8

03b1779

rickwierenga reviewed Jan 19, 2026

View reviewed changes

rickwierenga added 5 commits January 19, 2026 12:13

share code between functions

8f2775c

simplify well bottoms

9083918

ResponseParser class into functions; connection_state -> _connected

88a8422

make get_log_params lazy, delete _assign_params

270bdbe

remove override build_parameters when they dont add anything

75fc380

rickwierenga added 3 commits January 19, 2026 15:37

remove unused CommandMessage methods

c8f3549

HoiParams in CommandMessage

39b8c2c

no int(round(x)), just round(x)

efefd0b

delete HoiResponse, use HoiPacket instead

cccb111

rickwierenga force-pushed the nimbus8 branch from f3416cb to cccb111 Compare January 20, 2026 18:24

rickwierenga added 3 commits January 20, 2026 13:17

rename more parameters to match star

647fa41

type

d62bc16

add tests

60b9699

tl

ef86b40

rickwierenga mentioned this pull request Jan 21, 2026

Add binary parser #825

Merged

rickwierenga and others added 2 commits January 20, 2026 21:35

Merge main into nimbus8

fd0ba81

Added fixed offsets behavior for drop_tips in _compute_tip_handling_p…

1c46f45

…arameters Drop operations now use fixed offsets (10mm) instead of tip length, matching VantageBackend. Fixes regression from 8f2775c.

Nimbus Integration: TCP Backend and Interface Introspection #740

Are you sure you want to change the base?

Nimbus Integration: TCP Backend and Interface Introspection #740

Uh oh!

Conversation

cmoscy commented Nov 4, 2025

Uh oh!

cmoscy commented Nov 5, 2025

Uh oh!

rickwierenga commented Nov 5, 2025

Uh oh!

rickwierenga commented Nov 5, 2025

Uh oh!

cmoscy commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rickwierenga commented Nov 5, 2025

Uh oh!

cmoscy commented Nov 5, 2025

Uh oh!

cmoscy commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rickwierenga Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

cmoscy Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

rickwierenga commented Jan 19, 2026

Uh oh!

cmoscy commented Jan 20, 2026

Uh oh!

rickwierenga commented Jan 20, 2026

Uh oh!

cmoscy commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rickwierenga commented Jan 20, 2026

Uh oh!

rickwierenga commented Jan 20, 2026

Uh oh!

rickwierenga commented Jan 21, 2026

Uh oh!

cmoscy commented Jan 21, 2026

Uh oh!

cmoscy commented Jan 21, 2026

Uh oh!

rickwierenga commented Jan 21, 2026

Uh oh!

cmoscy commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cmoscy commented Nov 5, 2025 •

edited

Loading

cmoscy commented Jan 19, 2026 •

edited

Loading

cmoscy commented Jan 20, 2026 •

edited

Loading