Lucene Query (Usage Guide)

This page focuses on the practical usage of our Lucene query helpers when building a simple query builder or modifying user-entered queries.

Naming conventions

All utils functions are prefixed with luceneQuery so that every function doesn't need to be prefixed with lucene.

This also means if we want to migrate over the EQL functions that we can use the same names and prefix with eqlQuery.

Covered utils functions (from @skedulo/horizon-core):

import { luceneQuery } from '@skedulo/horizon-core'
const {
  parseQuery,
  stringifyQuery,
  appendNode,
  replaceNode,
  removeNode,
  isQueryUnsupportedInUi,
  parseQueryToRequestFilter,
} = luceneQuery

• parseQuery
• stringifyQuery
• appendNode
• replaceNode
• removeNode
• isQueryUnsupportedInUi
• parseQueryToRequestFilter

These utilities let you parse a query string into an AST, make edits to that AST, and convert it back to a string safely.

The AST is used by the Filters modal on List pages in Horizon Foundation. A filter is broken down into a set of nodes. The nodes represent each line in the Filter modal. Currently, the Filter modal only supports AND (not OR), so these nodes will all be joined back together as AND.

If it includes an OR filter or cannot be represented in the Filter modal, then the parsedQuery will be returned as null. You can check if a query is supported with the function isQueryUnsupportedInUi.

Quick Start

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery, stringifyQuery } = luceneQuery

// 1) Parse into a structured object
const query =
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'
const parsed = parseQuery(query)

// 2) Manipulate nodes how you wish
// Either appendNode, replaceNode or removeNode

// 3) Convert back to string when you're done
const output = parsed ? stringifyQuery(parsed.ast, parsed.dateOperations) : ''

Notes

• When passed a null AST, stringifyQuery returns an empty string: stringifyQuery(null, null) === ''.

parseQuery

Parses a Lucene query string into a structured object containing the top-level AST and a flat list of nodes.

Queries will always need to be parsed into nodes before making changes to the query.

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery } = luceneQuery

const parsed = parseQuery(
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*',
)

if (!parsed) {
  // parsing failed (invalid syntax)
}

// parsed.ast is a Lucene AST tree
// parsed.nodes is a flat list of Lucene nodes discovered in the AST

Tips

• If parseQuery returns null, the input was not valid Lucene syntax.
• The AST may contain implicit grouping nodes for multi-value fields.

Parsed AST example

Below is the AST shape produced by parsing the query.

parsed.ast

{
  "left": {
    "boost": null,
    "field": "filter_Job_JobStatus",
    "fieldLocation": {
      "end": { "column": 21, "line": 1, "offset": 20 },
      "start": { "column": 1, "line": 1, "offset": 0 }
    },
    "prefix": null,
    "quoted": false,
    "regex": false,
    "similarity": null,
    "term": "Queued",
    "termLocation": {
      "end": { "column": 29, "line": 1, "offset": 28 },
      "start": { "column": 22, "line": 1, "offset": 21 }
    }
  },
  "operator": "AND",
  "right": {
    "boost": null,
    "field": "filter_Job_Description",
    "fieldLocation": {
      "end": { "column": 55, "line": 1, "offset": 54 },
      "start": { "column": 33, "line": 1, "offset": 32 }
    },
    "prefix": null,
    "quoted": false,
    "regex": false,
    "similarity": null,
    "term": "*Breakfast*",
    "termLocation": {
      "end": { "column": 67, "line": 1, "offset": 66 },
      "start": { "column": 56, "line": 1, "offset": 55 }
    }
  }
}

parsed.nodes (flat list)

[
  {
    "boost": null,
    "field": "filter_Job_JobStatus",
    "fieldLocation": {
      "end": { "column": 21, "line": 1, "offset": 20 },
      "start": { "column": 1, "line": 1, "offset": 0 }
    },
    "prefix": null,
    "quoted": false,
    "regex": false,
    "similarity": null,
    "term": "Queued",
    "termLocation": {
      "end": { "column": 29, "line": 1, "offset": 28 },
      "start": { "column": 22, "line": 1, "offset": 21 }
    }
  },
  {
    "boost": null,
    "field": "filter_Job_Description",
    "fieldLocation": {
      "end": { "column": 55, "line": 1, "offset": 54 },
      "start": { "column": 33, "line": 1, "offset": 32 }
    },
    "prefix": null,
    "quoted": false,
    "regex": false,
    "similarity": null,
    "term": "*Breakfast*",
    "termLocation": {
      "end": { "column": 67, "line": 1, "offset": 66 },
      "start": { "column": 56, "line": 1, "offset": 55 }
    }
  }
]

stringifyQuery

Converts a parsed AST back to a Lucene string. Use this after editing the AST with appendNode/replaceNode/removeNode.

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery, stringifyQuery } = luceneQuery

const query =
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'
const parsed = parseQuery(query)
const result = parsed ? stringifyQuery(parsed.ast, parsed.dateOperations) : ''
// -> 'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'

appendNode

Appends a node to an existing AST with the specified boolean operator while preserving grouping.

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery, appendNode, stringifyQuery } = luceneQuery

const base =
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'
const parsed = parseQuery(base)
if (parsed) {
  const newNode = { field: 'filter_Job_Type', term: 'Cancelled', quoted: false }

  // Combine with AND (supported by simple builder)
  const nextAst = appendNode(parsed.ast, newNode)
  const output = stringifyQuery(nextAst, parsed.dateOperations)
  // -> 'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast* AND filter_Job_Type:Cancelled'
}

replaceNode

Replaces a specific node inside the AST with another node.

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery, replaceNode, stringifyQuery } = luceneQuery

const base =
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'
const parsed = parseQuery(base)

if (!parsed) {
  return base
}

const { ast, nodes, dateOperations } = parsed
const firstNode = nodes[0]
const replacement = {
  field: 'filter_Job_Type',
  term: 'Cancelled',
  quoted: false,
}
const nextAst = replaceNode(ast, firstNode, replacement)
const output = stringifyQuery(nextAst, dateOperations)
// -> 'filter_Job_Type:Cancelled AND filter_Job_Description:*Breakfast*'

removeNode

Removes a specific node from the AST, simplifying the tree when possible.

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQuery, removeNode, stringifyQuery } = luceneQuery

const base =
  'filter_Job_JobStatus:Queued AND filter_Job_Description:*Breakfast*'
const parsed = parseQuery(base)
if (parsed) {
  const { ast, nodes, dateOperations } = parsed
  const firstNode = nodes[0]
  const nextAst = removeNode(ast, firstNode)
  const output = stringifyQuery(nextAst, dateOperations)
  // -> 'filter_Job_Description:*Breakfast*'
}

isQueryUnsupportedInUi

This is used to detect any advanced queries that cannot be parsed as nodes for use in the Filters dropdown. Currently this includes any filters with OR in them and unknown comparators. See unit tests for more information.

const mockDataSchema = {
  fields: [], // Fields are not used in the function tested
  relationships: [],
  resourceId: 'd3d0879f-3029-4c15-9fa6-ecac176d4c0c',
}
const mockFieldOptions = [
  {
    value: 'filter_Job_Type',
    label: 'Type',
    isList: false,
    isNullable: true,
    isSortable: false,
    kind: Query.API.DataField_Kind.ENUM,
    isFilterable: true,
  },
  {
    value: 'filter_Job_Description',
    label: 'Description',
    isList: false,
    isNullable: true,
    isSortable: false,
    kind: Query.API.DataField_Kind.STRING,
    isFilterable: true,
  },
  {
    value: 'filter_Job_JobStatus',
    label: 'Status',
    isList: false,
    isNullable: false,
    isSortable: false,
    kind: Query.API.DataField_Kind.ENUM,
    isFilterable: true,
  },
]
const query = 'filter_Job_Type:Installation AND filter_Job_JobStatus:*Que*' // Enums don't support string matches
const result = isQueryUnsupportedInUi(
  parseQuery(query),
  mockDataSchema,
  mockFieldOptions,
)
// result -> true

Parse Lucene filter query string to JSON structure

parseQueryToRequestFilter

Typescript function which transforms a Lucene filter query string into the RequestFilter JSON structure replacing the transformation function in Platform Query API. This is exposed so that List Views can bypass Query API and use the connected function directly.

Quick example

import { luceneQuery } from '@skedulo/horizon-core'
const { parseQueryToRequestFilter } = luceneQuery

// Simple equality
const eq = parseQueryToRequestFilter('filter_Resource_UID:abc')
// => {
//   group: {
//     operator: 'AND',
//     conditions: [{ key: 'filter_Resource_UID', value: 'abc', comparison: 'EQUAL' }]
//   }
// }

// Wildcards map to STARTS_WITH / CONTAINS
const startsWith = parseQueryToRequestFilter('filter_Name:Foo*')
// => {
//   group: {
//     operator: 'AND',
//     conditions: [
//       { key: 'filter_Name', value: 'Foo*', comparison: 'STARTS_WITH' },
//     ],
//   },
// }
const contains = parseQueryToRequestFilter('filter_Name:*Foo*')
// => {
//   group: {
//     operator: 'AND',
//     conditions: [
//       { key: 'filter_Name', value: '*Foo*', comparison: 'CONTAINS' },
//     ],
//   },
// }

// NOTE: for $in[...] the value is a JSON stringified array (to match Query API expectations).
const inList = parseQueryToRequestFilter('filter_Status:["Draft","Published"]')
// => {
//   group: {
//     operator: 'AND',
//     conditions: [
//       {
//         key: 'filter_Status',
//         value: ['Draft', 'Published'],
//         comparison: 'IN',
//       },
//     ],
//   },
// }

// IN using $in[...] syntax (quotes optional). Hyphen prefix means NOT.
const notIn = parseQueryToRequestFilter('-filter_Code:$in["a","b","c"]')
// => {
//   group: {
//     operator: 'AND',
//     conditions: [
//       {
//         key: 'filter_Code',
//         value: '["a","b","c"]',
//         comparison: 'IN',
//         not: true,
//       },
//     ],
//   },
// }

// Grouping and boolean logic
const grouped = parseQueryToRequestFilter('(A:1 AND B:2) OR C:3')
// => {
//   group: {
//     operator: 'OR',
//     conditions: [
//       {
//         key: '',
//         value: '',
//         comparison: 'COMPARISON_UNSPECIFIED',
//         not: false,
//         group: {
//           operator: 'AND',
//           conditions: [
//             { key: 'A', value: 1, comparison: 'EQUAL' },
//             { key: 'B', value: 2, comparison: 'EQUAL' },
//           ],
//         },
//       },
//       { key: 'C', value: 3, comparison: 'EQUAL' },
//     ],
//   },
// }

Notes

• Throws an error when given an empty string.
• Supports NOT via either the NOT keyword or a leading hyphen before a key (e.g. -filter_IsDraft:true).
• Coerces booleans and numbers where applicable (e.g. true/false, 123.4).
• Supports escaped characters (e.g. 00052944\-df5e becomes 00052944-df5e in parsed values).
• IN value is parsed as a JSON stringified array
• For more examples, see comprehensive unit tests

Date parsing and ranges

The parser understands Lucene-style date ranges and NOW-based macros and converts them into one or two RequestFilter conditions, normalizing to UTC. The behavior below is defined by unit tests.

General rules

• A range written as [lower TO upper] or {lower TO upper} expands to two comparisons on the same key:
  • [ or ] means inclusive bound (>= for lower, <= for upper)
  • { or } means exclusive bound (> for lower, < for upper)
  • * can be used for an open bound
• Plain date strings (YYYY-MM-DD) are interpreted as entire-day ranges in the provided timezone.
• All outputs are ISO UTC strings
• You can pass a timezone (e.g., 'Australia/Brisbane')

Examples (using timezone 'Australia/Brisbane' and mock now '2025-12-23T14:00:00.000Z')

• Fixed date-time range (inclusive both ends):

  • Input: filter_Resource_CreatedDate:[2025-12-23T14:00:00.000Z TO 2025-12-24T13:59:59.999Z]
  • Output conditions:
    • { key: 'filter_Resource_CreatedDate', comparison: 'MORE_THAN_EQUAL', value: '2025-12-23T14:00:00.000Z' }
    • { key: 'filter_Resource_CreatedDate', comparison: 'LESS_THAN_EQUAL', value: '2025-12-24T13:59:59.999Z' }

• Is After (exclusive lower bound):

  • Input: filter_Resource_CreatedDate:{2025-12-24T05:08:04.088Z TO *]
  • Output: { key: 'filter_Resource_CreatedDate', comparison: 'MORE_THAN', value: '2025-12-24T05:08:04.088Z' }

• Is On Or After (inclusive lower bound):

  • Input: filter_Resource_CreatedDate:[2025-12-24T05:09:28.009Z TO *]
  • Output: { key: 'filter_Resource_CreatedDate', comparison: 'MORE_THAN_EQUAL', value: '2025-12-24T05:09:28.009Z' }

• Is Before (exclusive upper bound):

  • Input: filter_Resource_CreatedDate:[* TO 2025-12-24T05:10:03.368Z}
  • Output: { key: 'filter_Resource_CreatedDate', comparison: 'LESS_THAN', value: '2025-12-24T05:10:03.368Z' }

• Is On Or Before (inclusive upper bound):

  • Input: filter_Resource_CreatedDate:[* TO 2025-12-24T05:10:28.181Z]
  • Output: { key: 'filter_Resource_CreatedDate', comparison: 'LESS_THAN_EQUAL', value: '2025-12-24T05:10:28.181Z' }

• Specific date (entire day in timezone):

  • Input: filter_Resource_DateField:[2025-12-24 TO 2025-12-24]
  • Output conditions:
    • { key: 'filter_Resource_DateField', comparison: 'MORE_THAN_EQUAL', value: '2025-12-23T14:00:00.000Z' }
    • { key: 'filter_Resource_DateField', comparison: 'LESS_THAN_EQUAL', value: '2025-12-24T13:59:59.999Z' }

NOW macros

Use NOW:DATE as the base, optionally add +/- offsets and truncate to a period with /PERIOD. Common patterns in tests:

• Today:
  • filter_Resource_DateField:[NOW:DATE/DAY TO NOW:DATE+1DAY/DAY-1MILLISECOND]
  • -> start of local day to end of local day.
• Tomorrow:
  • filter_Resource_DateField:[NOW:DATE+1DAY/DAY TO NOW:DATE+2DAY/DAY-1MILLISECOND]
• Yesterday:
  • filter_Resource_DateField:[NOW:DATE-1DAY/DAY TO NOW:DATE/DAY-1MILLISECOND]
• Last/This/Next Week (weeks truncate with /WEEK):
  • Last Week: [...] [NOW:DATE-1WEEK/WEEK TO NOW:DATE/WEEK-1MILLISECOND]
  • This Week: [...] [NOW:DATE/WEEK TO NOW:DATE+1WEEK/WEEK-1MILLISECOND]
  • Next Week: [...] [NOW:DATE+1WEEK/WEEK TO NOW:DATE+2WEEK/WEEK-1MILLISECOND]
• Last/This/Next Month (truncate with /MONTH):
  • Last Month: [...] [NOW:DATE-1MONTH/MONTH TO NOW:DATE/MONTH-1MILLISECOND]
  • This Month: [...] [NOW:DATE/MONTH TO NOW:DATE+1MONTH/MONTH-1MILLISECOND]
  • Next Month: [...] [NOW:DATE+1MONTH/MONTH TO NOW:DATE+2MONTH/MONTH-1MILLISECOND]
• Last/This/Next Year (truncate with /YEAR):
  • Last Year: [...] [NOW:DATE-1YEAR/YEAR TO NOW:DATE/YEAR-1MILLISECOND]
  • This Year: [...] [NOW:DATE/YEAR TO NOW:DATE+1YEAR/YEAR-1MILLISECOND]
  • Next Year: [...] [NOW:DATE+1YEAR/YEAR TO NOW:DATE+2YEAR/YEAR-1MILLISECOND]

All the above produce UTC ISO outputs aligned with the provided timezone’s day/week/month/year boundaries, matching the unit tests in parse-query-to-request-filter.test.ts.

More advanced Lucene functions used only in the Filters modal

Below are very brief descriptions of commonly used helpers. For advanced usage and exact shapes, please refer to the unit tests in the lucene-query folder.

getFieldComparatorOptions

• Returns the list of comparator options (e.g., EQUALS, NOT_EQUALS, CONTAINS, IN, etc.) appropriate for a given field’s kind.

For example, for a STRING field, it will return:

const stringField = createMock<IFilterOption>({
  value: 'filter_Job_Description',
  label: 'Description',
  isList: false,
  isNullable: true,
  isSortable: false,
  kind: Query.API.DataField_Kind.STRING,
  isFilterable: true,
})

const result = getFieldComparatorOptions(stringField, false)

expect(result).toEqual([
  COMPARATOR_OPTIONS.equals,
  COMPARATOR_OPTIONS.doesNotEqual,
  COMPARATOR_OPTIONS.contains,
  COMPARATOR_OPTIONS.doesNotContain,
  COMPARATOR_OPTIONS.startsWith,
  COMPARATOR_OPTIONS.doesNotStartWith,
])

getFieldDefinition

• Builds the definition for the selected field type (e.g. Boolean, String) to drive the Filters UI.

For example, for a BOOL field, it will return:

const boolField = createMock<IFilterOption>({
  value: 'filter_Job_IsTemplate',
  label: 'Is Template',
  isList: false,
  isNullable: true,
  isSortable: false,
  kind: Query.API.DataField_Kind.BOOL,
  isFilterable: true,
})

const dataSchema = { fields: [], relationships: [], resourceId: 'x' }
const fieldDefinition = getFieldDefinition(boolField, dataSchema as any)

expect(fieldDefinition).toEqual({
  type: Query.API.DataField_Kind.BOOL,
  isList: false,
  valueOptions: [
    { value: 'true', label: 'Yes' },
    { value: 'false', label: 'No' },
  ],
  comparatorOptions: [
    COMPARATOR_OPTIONS.equals,
    COMPARATOR_OPTIONS.doesNotEqual,
  ],
  fieldRelationship: undefined,
})

getNode

• Creates a Lucene node object from a field, value, and comparator (e.g., EQUALS, CONTAINS, GREATER).
• Handles negated comparators by prefixing - to the field
• Handles string wildcards for contains/startsWith
• Handles range nodes for numeric/date comparisons
• Handles ONE_OF/NONE_OF chains, and IN/NOT_IN via an escaped $in payload.

For example, it will parse a CONTAINS field to add asterisks:

const node = getNode('Description', 'Breakfast', 'CONTAINS')
expect(node.field).toBe('Description')
expect(node.term).toBe('*Breakfast*')

parseNode

• Parses a Lucene node back into a simplified { fieldName, term, comparator } shape.
• Detects negation from -field
• Handles wildcards for CONTAINS/STARTS_WITH
• Handles range bounds for GREATER/LESS variants and BETWEEN
• Handles implicit OR chains for ONE_OF/NONE_OF, and IN/NOT_IN JSON payloads.

For example, it will parse a Lucene node and update the value as per field type.

const node = { field: '-Name', term: 'John' }
expect(parseNode(node)).toEqual({
  fieldName: 'Name',
  term: 'John',
  comparator: 'NOT_EQUALS',
})

getDefaultValueByComparator

• Provides default filter values per comparator and field type. Used to initialise filter inputs.
• For IN/NOT_IN - []
• For BETWEEN - ['', '']
• For date/datetime - today (seems incorrect)