Skip to content

Developers Guide

Jump to bottom
Zane Helton edited this page Jan 30, 2024 · 14 revisions

Building New Nodes

Welcome to the development guide for integrating new nodes into the system. This document provides a step-by-step process for developers to add new functionalities, ensuring a smooth integration with the existing infrastructure. If you aren't yet familiar, check out the Graph Lifecycle for a high-level overview of the system.

If you'd rather just see a commit, here's one for adding the nyt.top node.

⭐️ The first step is to think about what you'll call it. The convention is service.action for example twitter.post or volume.set so think about what your node interacts with and what it does. Simple nodes are easier for the model to work with, it also allows more flexibility.

Overview

Integrating a New Node

Integrating a new node into the system involves the following primary steps:

  1. Documenting the Node in the System Prompt
  2. Adding Fine-Tuning Examples
  3. Creating a New Node Class
  4. Registering the Node in the Node Map

Follow the detailed steps below to successfully integrate your new node.

Step 1: Documenting the Node in the System Prompt

To help the AI understand the purpose and usage of your new node, you need to document it in the SYSTEM_PROMPT variable located in ./src/graph/builder.py.

  1. Document Your Node: Add a concise and clear description of your node's functionality, inputs, and outputs. Use the existing nodes as a reference for the documentation format.

Example documentation for a new node:

SYSTEM_PROMPT = """
...
- youtube.search {query: String, shuffle: Boolean}: [Input: String, Boolean; Outputs: Array] Searches YouTube, outputs video list.
- youtube.play {video_id: String}: [Input: String] Plays YouTube video.
- sfx.play {video_id: String}: [Input: String] Plays sound effect for 5s.
- output.tts {text: String}: [Input: String] Play text to speech using ‘text’.
- volume.set {value: String}: [Input: String] Set/increase/decrease the volume. Only: values 0 thru 10, ‘+’, or ‘-‘.
- your.node {param1: Type, param2: Type}: [Input: Type, Type; Outputs: Type] Description of what your new node does.
...
"""
  1. Be Precise: Ensure the documentation accurately describes the functionality and the input/output data types.

Step 2: Adding Fine-Tuning Examples

Fine-tuning examples help the AI understand how to use the new node. Add your fine-tuning examples to ./fine_tune_data/builder.openai.jsonl.

  1. Format Examples: Each example should be in JSON Lines format. Here's the structure:
{"messages":[
    {"role":"system","content":""},
    {"role":"user","content":"Your user input"},
    {"role":"assistant","content":"Your expected AI response"}
]}

Unfortunately, for now, you'll need to craft your first few examples from scratch. But you can ask ChatGPT to create more examples. Omit the system prompt for faster results. Aim for at least 7 high quality examples to train the model thoroughly.

This step is optional, but recommended.

Example

{"messages":[{"role":"system","content":"SYSTEM_PROMPT"},{"role":"user","content":"Billy, please increase the volume."},{"role":"assistant","content":"{\"nodes\": {\"1\": {\"type\": \"input.voice\",\"text\": \"Billy, please increase the volume.\",\"outputs\": [\"2\"]},\"2\": {\"type\": \"volume.set\",\"value\": \"+\",\"outputs\": [\"3\"]},\"3\": {\"type\": \"done\"}}}"}]}

  1. Be Descriptive: Ensure that the examples cover various scenarios and use cases for the new node.

  2. Review: Double-check your examples for correctness and clarity.

Step 3: Creating a New Node Class

Create a new class for your node in ./src/graph/nodes. Use the code below as a starting point.

  1. Class Name: The class name should match the node's name (e.g., VolumeSetNode for volume.set).

  2. Inherit from ActionNode: Ensure your class inherits from the base ActionNode.

  3. Implement execute Method:

    • The execute method contains the core logic of the node.
    • Use input_data to handle inputs from previous nodes.
    • The method should return the node's output value.
    • Use self.data['...'] = '...' to add info for the model during the "personality" step. Can be used to add the result of a real-time query for example. Can also be useful for attaching data for Rabbit queues.

  4. Implement validate_inputs Method:

    • This method validates the inputs required for the node's operation.
    • Return True if the validation passes, otherwise False.
    • Not used currently.

  5. create_queue Attribute (Optional):

    • If your node interacts with external systems (like a Discord bot), set create_queue to True.
    • A RabbitMQ queue matching your node type will be created for external consumers that you may create.
      • You will call self.send_node_to_queue() within the execute function when ready.
    • Nodes that are handled internally don't usually need a queue.

Step 4: Registering the Node in the Node Map

Register your node in the node map located at ./src/graph/node_map.py.

  1. Import Your Node: At the top of node_map.py, import your node class.
from src.graph.nodes.your_own import YourOwnNode
  1. Add to Node Map:
  • Add an entry to the node map associating your node type with your node class.
NODE_MAP = {
    # ... existing nodes ...
    "your.node": YourOwnNode,
}

Further Tips

  1. Let it raise!: Don't stress about error handling, the code will retry automatically 3 times before failing.
  2. Check storage: There's a RabbitMQ queue on localhost containing the model in/out.
  3. Tip pending: I'll have to come back to this one.

By following these steps, you can seamlessly integrate new nodes into the system, enhancing its capabilities and maintaining its robustness.

Clone this wiki locally