Can't use multi-threading

[freeagent1384]

I've written some code using your library (very cool BTW) to confirm the MD5 hash matches the dcc download source before moving the file to my NAS. Because these are not speedy tasks I'm attempting to have a thread break off, do the work, and report back. I'm finding that a thread starts but does not complete. However, if I reload my script the thread continues for a bit and stops again. I can reload my script multiple times and it will eventually finish.

I'm thinking perhaps your plugin does not allow for multi-threading? Sample code provided below. After loading sample code, if you use the command "test" and provide a file name that exist in your dcc_download_path it will start calculating the MD5 of the file but it doesn't even execute to the first loop of the while clause. If you reload the script a few times you'll start to see in the log it continues to loop through while. If you quit IRSSI the remaining loops continue and the hash is calculated. Interestingly, if you uncomment line 51 and join the thread the script runs to completion.

Would love to hear your thoughts/feedback. I'm running on a Debian host with 4 CPU cores.

(sorry for all the edits. I could not get it to display the sample code correctly)

import irssi
import hashlib
import re
import logging
import os
import threading

def run_on_script_load() -> None:
    global dlPath
    dlPath = irssi.settings_get_str(b"dcc_download_path").decode("utf-8")
    if ( dlPath.__contains__("~") ):
        dlPath = dlPath.replace( "~", os.path.expanduser("~") )
    logLevel = "DEBUG"
    logging.basicConfig(filename="dccMediaLog.log", format="%(asctime)s %(levelname)s: %(message)s", filemode="w")
    global logger
    logger = logging.getLogger()
    logger.setLevel(logLevel)
    logger.debug("Download path set to %s." % (dlPath))

class md5chk:
    def __init__(self, BUF_SIZE=65536):
        self.buffSize = BUF_SIZE  #  Reading in 64kb chunks
    
    def fileChk(self, path, inFile) -> None:
        logger.debug("Calculating MD5")
        logger.debug("Line 26")
        md5 = hashlib.md5()
        logger.debug("Line 28")
        fullFile = os.path.join(path, inFile)
        logger.debug("Line 30")
        i = 0
        logger.debug("Line 32")
        with open(fullFile, "rb") as f:
            logger.debug("Line 34")
            while True:
                logger.debug("i: %s" % (i))
                data = f.read(self.buffSize)
                if not data:
                    break
                i += 1
                md5.update(data)
        return(format(md5.hexdigest()))


def testFunc(data, server, witem) -> None:
    pat = re.compile("(?P<fName>[^\s]+)")
    matches = pat.match(data.decode("utf-8"))
    fName = os.path.basename(matches.group("fName"))
    logger.debug("dccClose gave file: %s" % (fName))
    md5Matched = False
    logger.debug("Providing %s and %s to md5 calculator" % (dlPath, fName))
    md5Thread = threading.Thread(target=md5chk().fileChk, args=(dlPath, fName), daemon=False)
    md5Thread.start() 
#    fileMd5 = md5chk().fileChk(dlPath, fName)
    logger.debug("Thread started.")
#    md5Thread.join()
#    logger.debug("Thread finished")

irssi.command_bind(b"test", testFunc)

run_on_script_load()

[terminaldweller]

TL;DR unfortunately no.
Irssi is asynchronous, but single-threaded.

Irssi uses signals and callbacks to achieve asynchronicity. This doesn't mean that the python module can use async(you can use it but it won't be asynchronous) in python since we will need to have a custom event loop written for this module specifically that uses the irssi signals and callbacks and integrates into irssi's async implementation. The same thing applies to the use of threads.

You could use fork but then you'll be forking your entire irssi instance. You could give that a try and see if the result is satisfactory.

Out of curiosity, which version of python did you build the python module against?

[freeagent1384]

You are speedy on the response Sr. I'm running Python 3.11.2. I'm not a supper experienced developer but if there's anything I can to to help you with your project, please let me know.

[terminaldweller]

I'm not the developer/maintainer. I'm an irssi user just like you.
I've tested the module for python 3.12 and 3.13. both are working for my scripts. You can upgrade if you want to.
Keep building scripts for the python module. If anything comes up, please make an issue.