2.8 Quantexa Upgrade Guide

Quick Upgrade Overview
Community Upgrade Guidance
Recent Deprecations
Software Compatibility
Additional Information

Quick Upgrade Overview

The 2.8 Quantexa Upgrade consists of five main parts:

Dependencies and Platform Modernisation
User Interface (UI) Changes
Batch Resolver and Entity Resolution (ER)
Redis / Valkey Distributed Caching
Data Packs Migration

The majority of the dependency changes (Spark 3.5 migration, Cats Effect 3, Apache Pekko, library version bumps) are handled by automated Plop and Scalafix migrations via the Repository Tool. However, any custom code that deviates from best-practice patterns, particularly Cats Effect or Akka usage, will require manual migration. A new Quantexa license is also required before starting.

The User Interface tier sees the most user-facing change. Resource Viewer reaches General Availability (GA) and replaces several Investigation modules, Transaction Viewer has been fully removed (migrated to Data Viewer), Angular has been updated to version 18, and the web-app folder structure has been simplified. Projects should plan to test the UI thoroughly post-migration.

Batch Resolver receives several important changes: Network Generation and time-splitting have been removed, Classic mode is deprecated, and the Entity Attributes output model has changed. Filtered Compounds now use QSL instead of QEL. Most of these have Plop or Scalafix support, but projects with custom Batch Resolver scripts should budget additional effort.

Redis / Valkey distributed caching is introduced as optional in 2.8 but becomes mandatory from 2.9 onwards. Projects are strongly encouraged to set up Redis during the 2.8 upgrade to avoid a separate infrastructure change later.

Data Packs 2.3.x are compatible with Quantexa 2.8 and require Parsers 4.2.3 or later. Projects still on Parsers 3 must complete the Parsers 4 migration before upgrading to 2.8.

This page aims to provide additional guidance related to the 2.8 Quantexa Upgrade. For the full list of required migration steps, please refer to the Documentation site migration guide: 2.7 → 2.8 Upgrade Migration Guide.

Release Notes:

Community Release Announcement

2.8 Release Notes

Community Upgrade Guidance

Dependencies and Platform Modernisation

New Quantexa License Required

A security improvement to the Quantexa license means that existing licenses are not compatible with 2.8.0. You must request a new license before deploying the upgraded platform. This is a prerequisite that should be actioned early in the upgrade process to avoid blocking deployment.

Spark 3.5 Migration

Support for Spark 3.3 and 3.4, deprecated in 2.7, has been fully removed. Projects must now run on Spark 3.5. The BOM changes are applied automatically by Plop. The primary manual effort comes from ensuring custom project-owned Spark scripts are compatible with Spark 3.5 and upgrading Spark installations across all environments (local, CI/CD, and production). During internal migrations, no significant behaviour changes were found, so this should be a low-risk change for most projects.

Cats Effect 3

This is one of the more impactful dependency upgrades. The cats-effect, fs2, and fs2-Kafka libraries have all been updated to version 3.x. Code following the best-practice structure in Task Loading, Scoring, Alerting, Entity Store, and Data Source Integration Tests is migrated automatically. However, any custom code built upon earlier versions of these libraries, such as custom Kafka applications, must be manually migrated following the official Cats Effect 3 migration guide. Projects should use Project Example as a reference.

Akka Replaced by Apache Pekko

Due to CVEs raised against the Akka library, Quantexa has switched to Apache Pekko. This only affects deployments using app-graph-script. The migration is straightforward: ensure import akka.util.Timeout statements are changed to org.apache.pekko.util.Timeout. A Scalafix migration is available.

Other Library Upgrades

Several other libraries have been updated, all with Plop or Scalafix support:

Scalatest upgraded from 3.1.4 to 3.2.19
ScalaCheck upgraded (artifact name changed)
Chimney version updated
Java Faker replaced by Datafaker
ETL Util dependencies renamed (batch-common-api → spark-api, common-spark-utils → document-index-input-reader)
spark-excel artifact and package changed
spoiwo package updated from 1.x to 2.x
org.apache.commons.io must now be shaded in the Batch tier

JSON Configuration Validation

A notable change to be aware of is that configuration validation on application startup now causes the application to fail if Perspective, Bulk Search, or Foreign Document JSON configuration files contain unknown fields. Previously, these invalid fields were silently ignored. This means that configuration errors that have existed undetected in your project may now surface as startup failures. It is recommended to run the Configuration validation tests early in the upgrade to identify and fix any issues before attempting to deploy.

User Interface Changes

Resource Viewer (General Availability)

Resource Viewer is the most significant user-facing change in 2.8. It provides a unified, tabbed interface for viewing and interacting with Documents, Entities, and other resources. All projects upgrading to 2.8 will inherit the Resource Viewer, which replaces several existing Investigation modules including CompoundGraphViewModule, SourceViewerPluginModule, RecordViewerPluginModule, and others. These replaced modules are now deprecated and will be removed in the next major release.

This introduces breaking changes for deployments using Low-Code Configuration (LCC) or Record Viewer. Projects should follow the Resource Viewer Migration Guide and plan for testing time to verify the UI behaves as expected, particularly around any custom UI configuration.

Transaction Viewer Removal

Following its deprecation in 2.4, Transaction Viewer has been fully removed. Data Viewer is the recommended replacement, offering feature parity plus improvements such as no record limit, column-level UI filtering, and Scoring plugin integration. Projects still using Transaction Viewer must complete this migration. Follow the Transaction Viewer Migration Guide for guidance.

Angular 18 and web-app Simplification

The UI has been updated to Angular 18. Plop automatically handles the migration of your deployment's UI code. However, due to the many changes to NPM dependencies, Plop may fail to migrate web-app/package-lock.json to a consistent state. If you encounter UI build errors after migration, delete the node_modules and package-lock.json files from the web-app folder and run npm install to regenerate them.

The web-app folder structure has also been simplified, removing boilerplate files and simplifying how the application is bootstrapped. Tab configuration previously in MainViewComponent has moved to a dedicated tab-config.ts file.

Global UI Configuration Changes

The file structure used for global settings in Low-Code Configuration has changed. If your deployment uses global configuration loaded through JSON files on deployment, you must migrate to the new format. Follow the Global UI Configuration Migration Guide.

Batch Resolver and Entity Resolution

Network Generation and Time-splitting Removed

The Network build and Create ScoringGraph stages of Batch Resolver, which produce Natural Networks, have been removed following their deprecation in 2.7. Projects still using these must migrate to Graph Scripting QSL for network generation. The associated configuration options (appendFilteredDocuments, createNetworkStats, scoringGraphMaxNodeCount, etc.) must be removed from Batch Resolver configuration files.

Similarly, the time-splitting functionality has been removed. All timeSplit fields must be removed from Resolver JSON configuration.

Filtered Compounds: QEL to QSL Migration

The Quantexa Expression Language (QEL) expression field within Filtered Compound definitions has been replaced with a QSL where field. A Plop migration is available to translate existing QEL filters to QSL. Projects should review the translated filters to ensure correctness, particularly for complex expressions.

Additionally, Batch Resolver now warns if Filtered Compound definitions reference Record Attributes not defined in the input data. These warnings will be upgraded to errors in a future release, so projects should address them during the upgrade. Note that the default Address Resolution Template in the Data Packs core fragment contains an erroneous filteredCompoundDefinitions block referencing a rootCountyCode attribute; if present, this block should be removed.

Entity Attributes Output Model Changes

The excluded, subEntities, and linkExcluded fields have been removed from the Batch Resolver Entity Attributes output. A new failedDataQualityChecks field has been added. A Scalafix migration is available, but due to Scalafix limitations, imports referring to the removed SubEntity model must be removed manually.

Classic Mode Deprecated

Batch Resolver Classic mode is deprecated and will be removed in a future release. Projects should migrate to Resolver mode. To replicate previous Classic mode exclusions, use Element exclusions in ETL. See Migrating from Classic to Resolver mode for guidance.

Calling Batch Resolver and Graph Scripting QSL from Code Deprecated

Calling Batch Resolver and Graph Scripting QSL stages from code is deprecated and will be removed in a future major release. Projects must migrate to using the pre-defined scripts under com.quantexa.batchresolver.scripts and com.quantexa.graphqsl.scripts directly with spark-submit. If you previously used custom scripts to insert Run IDs from Metadata into file paths, use the updated dataPaths configuration.

Redis / Valkey Distributed Caching

Quantexa 2.8 introduces distributed caching with Redis or Valkey. While Redis is optional in 2.8, it becomes mandatory from 2.9 onwards. We strongly recommend configuring Redis during the 2.8 upgrade so that this infrastructure is in place before upgrading to 2.9.

The Repository Tool automatically adds a baseline application-redis-profiles.yml configuration file and updates docker-entrypoint scripts. Redis can be deployed via the Quantexa Helm Chart (Redis Sentinel), Google MemoryStore for QCP-hosted projects, or a manually deployed Redis/Valkey instance.

To enable Redis caching, set the following feature flag in your Spring configuration:

quantexa.caching.redis-caching.enabled: true

For full deployment guidance, see the 2.7 → 2.8 Migration Guide, section "Distributed caching using Redis or Valkey".

Data Packs Migration

Data Packs 2.3.x are compatible with Quantexa 2.8.x and require Parsers 4.2.3 or later.

Parsers 3 is no longer supported from Quantexa 2.8 onwards. Projects still on Parsers 3 must complete the migration to Parsers 4 before upgrading to 2.8. This migration should be treated as a separate workstream and is not recommended to be performed alongside the 2.8 platform upgrade. For planning and step-by-step guidance, see:

For the full Data Packs compatibility matrix, see Data Packs Compatibility Matrix.

Recent Deprecations

Batch Resolver Classic Mode

Classic mode is deprecated and will be removed in a future major release. Projects must migrate to Resolver mode. See Migrating from Classic to Resolver mode.

Elasticsearch 7 and OpenSearch 1

Elasticsearch 7 and OpenSearch 1 are deprecated in 2.8 and will be removed in 2.9. Projects should plan to upgrade to Elasticsearch 8 or OpenSearch 2.

Search 1

Search 1 is deprecated. Projects should migrate to Search 2.

Connect

Connect is deprecated in 2.8.

Quantexa Feature Engineering

Quantexa Feature Engineering is deprecated. Projects using version 0.9.0 must replace their imports with their own Spark transforms.

Graph Scripting DSL

Following its deprecation in 2.7, the DSL Conversion to Scoring Graph methods have been removed in 2.8. The Entity Statistical Profile Testing Framework's use of DSL models is also deprecated - projects must migrate to using the CalculatedEntityAttributes Batch Resolver output.

Parsers 3.x

As noted above, Parsers 3.x is no longer compatible with Quantexa 2.8. Projects should complete migration to Parsers 4.x before upgrading.