CI backend tests

[wouterhendriks]

Having trouble getting a backend test through CI.

Also tried locally by creating a "CI" WebHare that can be run with runkit @ci freshdbconsole. Normal tests run fine, but any test that tries to call TTLaunchApp crashes. Even the webhare_testsuite ones:

wh-ci runtest system.setup.test_userpreferences
[08:58:18.221] Test native system.setup.test_userpreferences
RunTestframework test_userpreferences job: -InlXs6CrPL63PbKHMW28w
Running testfw.prepare
Running whfs.prepare
Running testauthsettings
start wait
Call to OBJECT FUNCTION TTLAUNCHAPP() failed: Cannot resolve the URL 'testportal/' to a published folder
Started at:

/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:439:26: Called from INITEXPECT
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:661:25: Called from EXPECTSCREENCHANGE
/Users/wouter/projects/webhare/whtree/modules/webhare_testsuite/tests/system/setup/test_userpreferences.whscr:29:9: Called from TESTAUTHSETTINGS
 ** UNEXPECTED EXCEPTION: Cannot resolve the URL 'testportal/' to a published folder

/Users/wouter/projects/webhare/whtree/modules/publisher/lib/siteprofiles.whlib:47:11: Called from GETWEBCONTEXTFORURL
/Users/wouter/projects/webhare/whtree/modules/wrd/lib/auth.whlib:23:27: Called from GETWRDAUTHPLUGIN
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:1037:10: Called from GETTESTSUITEWRDAUTHPLUGIN
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:1250:124: Called from GETTESTCONTROLLER
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:1279:3: Called from TTLAUNCHAPP
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:466:11: Called from PROCESSCALLRESULT
/Users/wouter/projects/webhare/whtree/modules/tollium/lib/testframework.whlib:746:7: Called from EXPECTSCREENCHANGE
/Users/wouter/projects/webhare/whtree/modules/webhare_testsuite/tests/system/setup/test_userpreferences.whscr:29:3: Called from TESTAUTHSETTINGS
/Users/wouter/projects/webhare/whtree/modules/system/lib/testframework.whlib:1190:13: Called from TESTFRAMEWORK#RUNSERIE

Script failed: mod::webhare_testsuite/tests/system/setup/test_userpreferences.whscr (exitcode = 1)


+=====================================================================
| TEST RESULTS
+=====================================================================

The following modules failed:
 - system.setup.test_userpreferences
Test failed with exit code 1

Any idea how to fix this?

[unilynx]

Sounds like the WebHare backend doesn't have a URL. The environment variable WEBHARE_CI=1 must be set for WebHare to setup a test website during bootup (freshdbconsole) - you're probably missing that

This is how my CI Webhare is setup:

$HOME/whrunkit/ci/environment

WEBHARE_CI=1
WEBHARE_INITIAL_BACKEND=http://localhost:8888/

Executable file $HOME/whrunkit/ci/startup.sh

#!/bin/bash
echo "Setting up for tests"
wh users adduser --sysop --password secret sysop@example.net
wh registry set system.backend.layout.infotitle "CI login info"
wh registry set system.backend.layout.infotext "Login using sysop@example.net and secret"
exit 0

unfortunately the latter two registry keys currently don't do anything as startup texts work differently now and you can't easily set that up using the CLI

[wouterhendriks]

Hmm got this in /Users/wouter/whrunkit/ci/environment:

WEBHARE_CI=1
WEBHARE_INITIAL_BACKEND=http://localhost:8888/

Console:

[2026-02-03T14:56:42.635Z] Starting WebHare 5.9.0 (f89c78a6) in /Users/wouter/whrunkit/ci/whdata/ at http://127.0.0.1:23660

[unilynx]

That URL there is just the 'rescue' port and should always work to get a WebHare interface.

But you should be able to use it to figure out what the URL of the WebHare backend site is, and whether it indeed contains the 'testportal' because it sounds like it's not being setup for some reason

[wouterhendriks]

Thanks, got it working. Needed to do a couple of things:

1. Rename environment to environment.sh and make it executable, runkit only sources environment.sh (with .sh), not environment
2. Add export to the variable assignments in that file:

export WEBHARE_CI=1
export WEBHARE_INITIAL_BACKEND=http://localhost:8888/

3. Re-run runkit @ci freshdbconsole

The root cause: without the environment variables being passed, initdb.whscr skips the webserver/port setup block (the WEBHARE_INITIAL_BACKEND / WEBHARE_CI check around line 278), so the backend site gets created as a folder but without an output webserver.

[unilynx]

Rename environment to environment.sh and make it executable, runkit only sources environment.sh (with .sh), not environment

Just to correct here, this should not have been needed and 'environment' works for me. See also runkit's documentation

  - `environment` - if present and executable, this will be added to the environment. This syntax is supported by both container and direct execution, unlike environment.sh
  - `environment.sh` - if present and executable, this will be sourced just before executing the command. Use this to setup eg. `export WEBHARE_CI=1`

It's true that the file isn't sourced, but it's not like that's the only way to set an environment variable

(only error in the documentation there is that environment doesn't have to executable)