How to boost Lucene queries with field values?

[semerzo]

Hello,

I am curious how to boost full-text queries with document (node) values. We do get a field from our source systems that contain a weight that we shall use as multiplicator.

Lucene allows to influence the scores via boosting. This could be done via index or query time boosting. But my problem is that this shall not be done via a flat factor, but by a custom weight that is individual per document.

I have found this blog post:

GraphAware

Deep Dive into Neo4j 3.5 Full Text Search | GraphAware

In this blog we will go over the Full Text Search capabilities available in the latest major release of Neo4j.Contrary to our usual blogs, the content w...

In there you can find how to boost with a flat value. Unfortunately that did not help me with my issue.

[stefan_armbrust]

When doing fulltext index queries via CALL db.index.fulltext.queryNodes(<indexName>, <luceneQuery>) YIELD node, score RETURN node.title, score
you can pass in a lucene query as second argument. For details on lucene query syntax see https://lucene.apache.org/core/5_5_5/queryparser/org/apache/lucene/queryparser/classic/package-summa....

Assume you have two properties title and description and you want to search for liar boosting results on title you could do

call db.index.fulltext.queryNodes(myindex, "title:liar^4 OR description:liar") yield node, score return node, score

[semerzo]

Thank you Stefan, this is what I meant when: I found out how to boost it with a flat value

We do get for each record an individual weight that we shall take into account. So image a document has three attributes

• title
• description
• weight

Document-1 would have the following values

• title: Not telling lies
• description: Trust the author
• weight: 1

Document-2 would have the following values

• title: Another title
• description: Some great description
• weight: 3

I would like to perform this

call db.index.fulltext.queryNodes(myindex, "title:liar^weight OR description:liar") yield node, score return node, score

[stefan_armbrust]

so you want the weight property being a dynamic boost factor on a per-record level?

[semerzo]

Exactly

[stefan_armbrust · Accepted solution]

Lucene does support that, see https://lucene.apache.org/core/5_5_5/core/org/apache/lucene/search/package-summary.html#scoring and https://lucene.apache.org/core/5_5_5/core/org/apache/lucene/document/Field.html#setBoost-float-

However that is currently not used by Neo4j fulltext indexing and would require a change in Neo4j itself. Maybe create a github issue for this and reference this thread there.

[tonny008]

index the weight field and in query list all the weight like below
call db.index.fulltext.queryNodes(myindex, "(title:liar OR description:liar) AND (weight:3^3 OR weight:2^2) weight:1^1") yield node, score return node, score

[EarthlingDavey]

Old question, I know...
I stumbled on this blog post, Exploring the full-text search index in Neo4j on a movies dataset, where it's written:

let’s combine the rating-boosting and the time decay effect in a single query. And just so you know you can always manipulate results later with cypher, let’s add a filter that will return only thrillers.

WITH apoc.text.join([x in range(0,10) | 
"string_release_date:" + toString((date().year — x)) + "^" +   
  toString(10-x)]," ") as time_decay
CALL db.index.fulltext.queryNodes("MovieIndex", "title:dream string_rating:[50 TO 99]^2 "+ time_decay) YIELD node, score
// filter only thrillers
MATCH (node)-[:HAS_TAG]->(:MovieTag{id:'thriller'})
RETURN node.title as title,score
LIMIT 5

It looks like the author is using a node property as a multiplier for boost.