Skip to content

Several updates #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FormShareDev wants to merge 22 commits into
base: master
Choose a base branch
from
Open

Several updates #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1917d57
Update STATA export
Dec 11, 2023
ec2a67e
Separare labels in different file
Dec 11, 2023
929cb89
Report the sheet in the XLSX with their table name
Feb 11, 2024
a7ee818
Fix merging error when PK is date or datetime
Feb 13, 2024
d5f9e45
Fix error when a form has only external selects
Mar 26, 2024
bd218d9
Control spaces in external multi-selects
Apr 18, 2024
e216a5e
Fix bug when lookup codes has extra spaces
Aug 16, 2024
1920107
Update libraries. Remove MySQLDenormalize as we are working on a new …
Sep 8, 2024
c0a6ac5
Start compatibility between Qt5 and Qt6
Oct 17, 2024
2a955f3
More fixes for Qt5-Qt6 compatibility
Oct 18, 2024
09bb2b4
Allow two or more external CSVs with same data
Nov 4, 2024
58aeb94
Fix bug with qt6
Nov 11, 2024
3f4126d
Replace CSV code with external tools
Dec 10, 2024
6c68ca1
Clear sqls
Dec 11, 2024
2348a0d
Dont infere type
Dec 11, 2024
995d86a
Report error
Dec 16, 2024
8aa101c
Fix bug when addind a new repeat with multi-selects
Sep 21, 2025
15f8d60
Fix bug
Sep 21, 2025
314a5b0
Overwrite ON DELETE if its different in the XML
Feb 16, 2026
bf10647
Fix JXFormToMysql
Feb 17, 2026
0e19a55
Bump Quazip to 1.5
Apr 7, 2026
e6cad70
Add AI license note
Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
*.*#
core
!core/
.idea/
tags
.DS_Store
*.debug
Expand Down
52 changes: 18 additions & 34 deletions AWS_build_script.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
#ODK Tools - Installation script tested in Ubuntu server 20.04
#ODK Tools - Installation script tested in Ubuntu server 22.04

sudo add-apt-repository universe
sudo add-apt-repository multiverse
sudo apt-get update
sudo apt-get install build-essential qt5-default qtbase5-private-dev qtdeclarative5-dev libqt5sql5-mysql libqt5sql5-sqlite cmake mongodb jq libboost-all-dev unzip zlib1g-dev automake npm sqlite3 libqt5sql5-sqlite golang-go

sudo wget https://dev.mysql.com/get/mysql-apt-config_0.8.17-1_all.deb
sudo dpkg -i ./mysql-apt-config_0.8.17-1_all.deb
wget https://dev.mysql.com/get/mysql-apt-config_0.8.32-1_all.deb
sudo dpkg -i ./mysql-apt-config_0.8.32-1_all.deb

sudo apt-get update
sudo apt-get install mysql-shell

sudo apt-get install -y build-essential qtbase5-dev qtbase5-private-dev qtdeclarative5-dev libqt5sql5-mysql cmake jq libboost-all-dev unzip zlib1g-dev automake libmysqlclient-dev mysql-client sqlite3 libqt5sql5-sqlite golang-go mysql-shell openjdk-17-jre-headless

cd /opt

sudo wget https://github.com/BurntSushi/xsv/releases/download/0.13.0/xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz
cd /bin
sudo tar xvfz /opt/xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz
sudo tar xvfz xsv-0.13.0-x86_64-unknown-linux-musl.tar.gz
sudo cp xsv /bin

cd /opt

Expand All @@ -26,46 +26,30 @@ sudo cp csv2xlsx /bin
sudo git clone https://github.com/qlands/odktools.git
sudo mkdir odktools-deps
cd odktools-deps
sudo wget https://github.com/mongodb/mongo-c-driver/releases/download/1.21.1/mongo-c-driver-1.21.1.tar.gz
sudo wget https://github.com/mongodb/mongo-cxx-driver/releases/download/r3.6.7/mongo-cxx-driver-r3.6.7.tar.gz
sudo wget https://github.com/jmcnamara/libxlsxwriter/archive/refs/tags/RELEASE_1.1.4.tar.gz
sudo wget https://github.com/stachenov/quazip/archive/refs/tags/v1.3.tar.gz

sudo wget https://github.com/jmcnamara/libxlsxwriter/archive/refs/tags/v1.1.8.tar.gz
sudo wget https://github.com/stachenov/quazip/archive/refs/tags/v1.4.tar.gz

sudo git clone https://github.com/rgamble/libcsv.git

sudo tar xvfz mongo-c-driver-1.21.1.tar.gz
cd /opt/odktools-deps/mongo-c-driver-1.21.1
sudo mkdir build_here
cd /opt/odktools-deps/mongo-c-driver-1.21.1/build_here
sudo cmake ..
sudo make
sudo make install
cd /opt/odktools-deps

sudo tar xvfz mongo-cxx-driver-r3.6.7.tar.gz
cd /opt/odktools-deps/mongo-cxx-driver-r3.6.7
sudo mkdir build_here
cd /opt/odktools-deps/mongo-cxx-driver-r3.6.7/build_here
sudo cmake -DCMAKE_C_FLAGS:STRING="-O2 -fPIC" -DCMAKE_CXX_FLAGS:STRING="-O2 -fPIC" -DBSONCXX_POLY_USE_BOOST=1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local ..
sudo make
sudo make install
cd /opt/odktools-deps


sudo tar xvfz v1.3.tar.gz
cd /opt/odktools-deps/quazip-1.3
sudo tar xvfz v1.4.tar.gz
cd /opt/odktools-deps/quazip-1.4
sudo mkdir build
cd /opt/odktools-deps/quazip-1.3/build
cd /opt/odktools-deps/quazip-1.4/build
sudo cmake -DCMAKE_C_FLAGS:STRING="-fPIC" -DCMAKE_CXX_FLAGS:STRING="-fPIC" ..
sudo make
sudo make install
cd /opt/odktools-deps

sudo ln -s /usr/bin/aclocal-1.15 /usr/bin/aclocal-1.14
sudo ln -s /usr/bin/automake-1.15 /usr/bin/automake-1.14
sudo ln -s /usr/bin/aclocal-1.16 /usr/bin/aclocal-1.14
sudo ln -s /usr/bin/automake-1.16 /usr/bin/automake-1.14

sudo tar xvfz RELEASE_0.7.6.tar.gz
cd libxlsxwriter-RELEASE_0.7.6
sudo tar xvfz v1.1.8.tar.gz
cd libxlsxwriter-1.1.8
sudo mkdir build
cd build
sudo cmake ..
Expand Down
42 changes: 30 additions & 12 deletions JSONToMySQL/mainclass.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@ mainClass::mainClass(QObject *parent) : QObject(parent)

void mainClass::run()
{
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
sqlStream.setCodec("UTF-8");

#endif
recordMap = QDomDocument("ODKRecordMapFile");
recordMapRoot = recordMap.createElement("ODKRecordMapXML");
recordMapRoot.setAttribute("version", "1.0");
Expand Down Expand Up @@ -217,7 +218,9 @@ void mainClass::run()
if (XMLLogFile.open(QIODevice::WriteOnly | QIODevice::Text))
{
QTextStream strXMLLog(&XMLLogFile);
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
strXMLLog.setCodec("UTF-8");
#endif
xmlLog.save(strXMLLog,1,QDomNode::EncodingFromTextStream);
XMLLogFile.close();
}
Expand Down Expand Up @@ -246,9 +249,9 @@ void mainClass::run()
}
UUIDFile.close();

QFileInfo fi(json);
//QFileInfo fi(json);
// Inserting into the submission database
QString sql;
//QString sql;

if (outputType == "h")
{
Expand Down Expand Up @@ -276,7 +279,9 @@ void mainClass::run()
if (file.open(QIODevice::WriteOnly | QIODevice::Text))
{
QTextStream out(&file);
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
out.setCodec("UTF-8");
#endif
recordMap.save(out,1,QDomNode::EncodingFromTextStream);
file.close();
}
Expand Down Expand Up @@ -799,7 +804,11 @@ QList<TfieldDef > mainClass::createSQL(QSqlDatabase db, QVariantMap jsonData, QS

mSelectValues.clear();
//Split the values into a stringlist
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
mSelectValues.append(insertObject.itemValue(pos).simplified().split(" ",QString::SkipEmptyParts));
#else
mSelectValues.append(insertObject.itemValue(pos).simplified().split(" ",Qt::SkipEmptyParts));
#endif
//Process each value
for (nvalue = 0; nvalue < mSelectValues.count();nvalue++)
{
Expand Down Expand Up @@ -1082,7 +1091,11 @@ void mainClass::processLoop(QJsonObject jsonData, QString loopTable, QString loo
QString fieldValue;
QString xmlKey = loopXMLRoot + "/" + loopItems[iItem] + "/" + fields[pos].xmlCode;
fieldValue = jsonData.value(xmlKey).toString("");
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
QStringList parts = fieldValue.split(" ",QString::SkipEmptyParts);
#else
QStringList parts = fieldValue.split(" ",Qt::SkipEmptyParts);
#endif
for (int ipart = 0; ipart < parts.count(); ipart++)
{
recordUUID=QUuid::createUuid();
Expand Down Expand Up @@ -1169,7 +1182,11 @@ int mainClass::procTable2(QSqlDatabase db,QJsonObject jsonData, QDomNode table,
loopXMLRoot = table.toElement().attribute("xmlcode","none");
QString loopTable;
loopTable = table.toElement().attribute("mysqlcode","none");
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
QStringList loopItems = table.toElement().attribute("loopitems","").split(QChar(743),QString::SkipEmptyParts);
#else
QStringList loopItems = table.toElement().attribute("loopitems","").split(QChar(743),Qt::SkipEmptyParts);
#endif
if (loopItems.count() > 0)
{
QDomNode fieldNode = table.firstChild();
Expand All @@ -1180,7 +1197,7 @@ int mainClass::procTable2(QSqlDatabase db,QJsonObject jsonData, QDomNode table,
{
TfieldDef aLoopField;
aLoopField.xmlCode = fieldNode.toElement().attribute("xmlcode","");
aLoopField.decSize = fieldNode.toElement().attribute("decsize","0").toInt();
aLoopField.decSize = QString::number(fieldNode.toElement().attribute("decsize","0").toInt());
aLoopField.key = false;
if (fieldNode.toElement().attribute("isMultiSelect","false") == "true")
{
Expand All @@ -1191,7 +1208,7 @@ int mainClass::procTable2(QSqlDatabase db,QJsonObject jsonData, QDomNode table,
aLoopField.multiSelect = false;
aLoopField.name = fieldNode.toElement().attribute("mysqlcode","");
aLoopField.ODKType = fieldNode.toElement().attribute("odktype","");
aLoopField.size = fieldNode.toElement().attribute("size",0).toInt();
aLoopField.size = QString::number(fieldNode.toElement().attribute("size",0).toInt());
aLoopField.type = fieldNode.toElement().attribute("type","varchar");
loopFields.append(aLoopField);
}
Expand All @@ -1216,8 +1233,8 @@ int mainClass::procTable2(QSqlDatabase db,QJsonObject jsonData, QDomNode table,
field.xmlCode = child.toElement().attribute("xmlcode");
field.type = child.toElement().attribute("type","varchar");
field.ODKType = child.toElement().attribute("odktype","text");
field.size = child.toElement().attribute("size","0").toInt();
field.decSize = child.toElement().attribute("decsize","0").toInt();
field.size = QString::number(child.toElement().attribute("size","0").toInt());
field.decSize = QString::number(child.toElement().attribute("decsize","0").toInt());

if (child.toElement().attribute("key").toStdString() == "true")
field.key = true;
Expand Down Expand Up @@ -1245,15 +1262,16 @@ int mainClass::procTable2(QSqlDatabase db,QJsonObject jsonData, QDomNode table,
keys.append(createSQL(db,jsonData.toVariantMap(),tableCode,fields,keys,emptyMap,true)); //Change the variant map to an object later on!
genSQL = false;
}
if ((osm == false) && (loop == false))
if ((osm == "false") && (loop == "false"))
{
QJsonArray children = jsonData.value(child.toElement().attribute("xmlcode")).toArray();
for (int chld = 0; chld < children.count(); chld++)
procTable2(db,children.at(chld).toObject(),child,keys);
procTable2(db,jsonData,child,keys);

}
else
{
procTable2(db,jsonData,child,keys);
QJsonArray children = jsonData.value(child.toElement().attribute("xmlcode")).toArray();
for (int chld = 0; chld < children.count(); chld++)
procTable2(db,children.at(chld).toObject(),child,keys);
}
}
else
Expand Down
7 changes: 6 additions & 1 deletion JXFormToMysql/jxformtomysql.pro
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ CONFIG -= app_bundle
TEMPLATE = app

INCLUDEPATH += ../3rdparty
LIBS += -lquazip1-qt5 -lcsv

greaterThan(QT_MAJOR_VERSION, 5) {
LIBS += -lquazip1-qt6 -lcsv
QT += core5compat
} else {
LIBS += -lquazip1-qt5 -lcsv
}
SOURCES += main.cpp
Loading