Competency Search with Ors and Sort by Relevance Doesn't Work

[jeannekitchens]

place holder

[sedula27]

Search by relevance returns less accurate results, while the default order provides more reliable matches.

{
	"Query": {
		"@type": "ceasn:CompetencyFramework",
		"search:termGroup": {
			"search:value": [
				{
					"search:operator": "search:orTerms",
					"^ceasn:isPartOf": {
						"search:operator": "search:orTerms",
						"ceasn:competencyLabel": "educational leadership competencies for ai draft ",
						"ceasn:competencyText": "educational leadership competencies for ai draft ",
						"ceasn:comment": "educational leadership competencies for ai draft ",
						"ceasn:conceptKeyword": "educational leadership competencies for ai draft ",
						"ceasn:codedNotation": "educational leadership competencies for ai draft "
					},
					"ceasn:creator": {
						"ceterms:subjectWebpage": "educational leadership competencies for ai draft "
					},
					"ceasn:publisher": {
						"ceterms:subjectWebpage": "educational leadership competencies for ai draft "
					},
					"ceasn:name": "educational leadership competencies for ai draft ",
					"ceasn:description": "educational leadership competencies for ai draft ",
					"ceasn:source": "educational leadership competencies for ai draft ",
					"ceasn:conceptKeyword": "educational leadership competencies for ai draft "
				}
			],
			"search:operator": "search:andTerms"
		}
	},
	"Skip": 0,
	"Take": 5,
	"Sort": "^search:relevance"
}

Query with out sort order

{
	"Query": {
		"@type": "ceasn:CompetencyFramework",
		"search:termGroup": {
			"search:value": [
				{
					"search:operator": "search:orTerms",
					"^ceasn:isPartOf": {
						"search:operator": "search:orTerms",
						"ceasn:competencyLabel": "educational leadership competencies for ai draft ",
						"ceasn:competencyText": "educational leadership competencies for ai draft ",
						"ceasn:comment": "educational leadership competencies for ai draft ",
						"ceasn:conceptKeyword": "educational leadership competencies for ai draft ",
						"ceasn:codedNotation": "educational leadership competencies for ai draft "
					},
					"ceasn:creator": {
						"ceterms:subjectWebpage": "educational leadership competencies for ai draft "
					},
					"ceasn:publisher": {
						"ceterms:subjectWebpage": "educational leadership competencies for ai draft "
					},
					"ceasn:name": "educational leadership competencies for ai draft ",
					"ceasn:description": "educational leadership competencies for ai draft ",
					"ceasn:source": "educational leadership competencies for ai draft ",
					"ceasn:conceptKeyword": "educational leadership competencies for ai draft "
				}
			],
			"search:operator": "search:andTerms"
		}
	},
	"Skip": 0,
	"Take": 5
}```

[jeannekitchens]

@excelsior @edgarf we had found the a-z and z-a search was never implemented. Maybe this isn't implemented?

[excelsior]

@jeannekitchens It was implemented (and still is, depending on the query's conditions), this is a side effect on the optimizations we did in June. I'll bring it back the right soon!

[excelsior]

@jeannekitchens I found the following problem with ordering by relevancy. This type of ordering uses the rank of a full-text match to position a resource in the output.

The problem is a query can contain more than one full-text condition. E.g. the query shared by Sneha contains 11 (!) such conditions. A single resource can have multiple matches, each with different rank. Thus it's not clear which of those ranks to use when ordering the results. If we use a sum of the ranks, the results may appear very random.

Should we prioritize the first condition if there're more than one or add an extra parameter to the API enable to specifiy the prefered property?

[jeannekitchens]

@excelsior this query example is ceasn:Competency properties and ceasn:CompetencyFramework properties. Depending on if this is for the Framework or Competency search , should that factor in to prioritizing the conditions?

@sedula27 which search is this query used for?

@excelsior also, are you saying only 1 property can be prioritized or all of the properties e.g., with a class?

[excelsior]

@jeannekitchens Yes, we can only use one property when ordering the results. And we can (and should, I believe) prioritize the main resource being searched, i.e. the CompetencyFramework in this case.

[jeannekitchens]

@sedula27 do we need to discuss this query further? See comments from Alex.