Items added to FreeFields have extra CR appended to RegEx Data extract

[etiitservices]

This is an issue there has been knocking around for a few years now, we noticed it first in 2022 and it breaks Acknowledgements being sent back into Nagios as well as alert matching if you update anything in the ticket's freefields window. This was a forum post I raised about it back then https://community.znuny.org/viewtopic.php?t=43354 (znuny 6.4.2/SM 6.4.2) and we finally got round to diagnosing the issue properly this week.

Now running Znuny 7.0.19 with SystemMonitoring 7.0.2 and the issue is still the same.

When an alert is retrieved via email and processed correctly in the SystemMonitoring filter the function that puts the Host and Service Items into the Ticket FreeFields items adds a CR (0D in hex) to the end of each data item.

We've validated the RegEx used at regex101.com and there is no Carriage Return in the capture groups returned:

-- Test data --

***** Nagios *****

Notification Type: PROBLEM

Service: MGMT
Host: testserver
Host Alias: Dev C7 Test Server
Address: 111.222.333.444
State: CRITICAL

Date/Time: Fri Oct 28 17:10:41 BST 2022

Additional Info:

connect to address 111.222.333.444 and port 80: Connection refused

-- Test Data End --

-- RegEx strings --

HostRegExp -> \s*Host:\s+(.*)\s*
ServiceRegEx -> \s*Service:\s+(.*)\s*

--

Links to RegEx examples:
https://regex101.com/r/KbXp2p/1
https://regex101.com/r/glSNKp/1

We'd therefore guess that the code AFTER the regex parsing has captured the retrieved data correctly is then adding a CR itself at some point.

If you look at the FreeFields data in the created ticket, make NO CHANGES to anything, and just click save the extra CRs are stripped from the FreeFields. Subsequent SystemMonitoring updates to the ticket, for example a recovery email, no longer match up as there is no longer an CR at the end of the FreeFields ticket data - you can see this in the communication logs quite clearly.

[markdiss]

and still the same issue with SystemMonitoring 7.1.2 running on Znuny 7.1.7

[markdiss]

If there is anything else I can provide to help troubleshoot this please let me know.
Issue still present on Znuny 7.2.3 and SystemMonitoring 7.2.1
Backend Rocky Linux 9.7, MariaDB 10.11.15, perl 5.32.1

[root@znuny ~]# /opt/znuny/bin/znuny.CheckModules.pl
o Apache::DBI......................ok (v1.12)
o Apache2::Reload..................ok (v0.13)
o Archive::Tar.....................ok (v2.38)
o Archive::Zip.....................ok (v1.68)
o Authen::SASL.....................ok (v2.16)
o Crypt::Eksblowfish::Bcrypt.......ok (v0.009)
o Crypt::JWT.......................Not installed! To install, you can use: 'cpan Crypt::JWT'. (optional - JSON web token support.)
o Crypt::OpenSSL::X509.............Not installed! To install, you can use: 'cpan Crypt::OpenSSL::X509'. (optional - X509 certificate support.)
o CSS::Minifier::XS................ok (v0.13)
o Data::UUID.......................ok (v1.226)
o Date::Format.....................ok (v2.24)
o DateTime.........................ok (v1.54)
o DateTime::TimeZone.............ok (v2.62)
o DBI..............................ok (v1.643)
o DBD::mysql.......................ok (v4.050)
o DBD::ODBC........................Not installed! To install, you can use: 'cpan DBD::ODBC'. (optional - Required to connect to a MS-SQL database.)
o DBD::Oracle......................Not installed! To install, you can use: 'cpan DBD::Oracle'. (optional - Required to connect to a Oracle database.)
o DBD::Pg..........................Not installed! To install, you can use: 'cpan DBD::Pg'. (optional - Required to connect to a PostgreSQL database.)
o Digest::SHA......................ok (v6.02)
o Encode::HanExtra.................ok (v0.23)
o Excel::Writer::XLSX..............ok (v0.95)
o Hash::Merge......................ok (v0.302)
o iCal::Parser.....................ok (v1.21)
o IO::Socket::SSL..................ok (v2.073)
o JavaScript::Minifier::XS.........ok (v0.15)
o Jq...............................Not installed! To install, you can use: 'cpan Jq'. (optional - Support for extended condition checking via Jq for the generic interface.)
o JSON::XS.........................ok (v4.04)
o List::Util::XS...................ok (v1.56)
o LWP::UserAgent...................ok (v6.53)
o Mail::IMAPClient.................ok (v3.42)
o IO::Socket::SSL................ok (v2.073)
o Authen::NTLM...................ok (v1.09)
o ModPerl::Util....................ok (v2.000012)
o Moo..............................ok (v2.005004)
o Net::DNS.........................ok (v1.29)
o Net::LDAP........................ok (v0.68)
o Net::LDAP::Constant..............ok (v0.24)
o Net::SMTP........................ok (v3.13)
o Spreadsheet::XLSX................ok (v0.18)
o Template.........................ok (v3.009)
o Template::Stash::XS..............ok (undef)
o Text::Diff::FormattedHTML........ok (v0.08)
o Text::CSV_XS.....................ok (v1.46)
o Time::HiRes......................ok (v1.9764)
o Time::Piece......................ok (v1.3401)
o XML::LibXML......................ok (v2.0206)
o XML::LibXSLT.....................ok (v1.99)
o XML::Parser......................ok (v2.46)
o YAML::XS.........................ok (v0.82)

[hanneshal]

Hi @markdiss
we need you sample config, ticket histories and sample mails (initial, update | closing event).
There is nothing appended to the strings after the processing. In the most cases it is a simple misconfiguration.

But we never know what the real cause is until we get more information.
So drop them here and we will have a look.
See #2 as an example.

Regards
Johannes