Planning for Success
Planning for success with a Quantexa Platform upgrade is no different from planning for success in any other project. A well-defined approach, clear scope, accurate estimates, and appropriate resourcing are the essential foundations for creating an upgrade plan that is set up to succeed. This guide provides a comprehensive, step-by-step framework to help you navigate the process from start to finish. Who is this guide for? This guide is primarily intended for Project Managers, Technical Leads (TLs), and Delivery Owners who are responsible for planning and executing a Quantexa Platform upgrade. Business stakeholders and solution designers will also find the sections on scoping and testing requirements valuable. What will you learn? By following this guide, you will have the tools and knowledge to: Clearly define the goals and scope of your upgrade. Choose the right upgrade strategy for your specific circumstances. Accurately estimate the effort and resources required. Assemble a comprehensive and actionable final upgrade plan. Before you begin We assume you have a foundational understanding of your organization's current Quantexa implementation and general project management principles. Familiarity with your platform's architecture and existing use cases will be highly beneficial. The following pages provide a step-by-step guide on how to best plan an upgrade of your Quantexa Platform instance. The Upgrade Planning Process The following pages provide a step-by-step guide on how to best plan an upgrade of your Quantexa Platform instance. We have broken the process down into key stages to help you navigate the journey. Stage 1: Setting the Direction This initial stage is about defining the high-level goals and approach for your upgrade. Page Description Identify Your Destination Understand the target version of the platform and the key benefits it brings. Define Your Upgrade Approach Choose the right strategy, whether it's a "like-for-like" or a transformational upgrade. Planning A Multi-Use Case Platform Upgrade Learn the specific considerations for planning an upgrade on a platform that serves multiple business areas. Stage 2: Defining the Work This stage focuses on the detailed work of understanding the scope and effort involved. Page Description Define Upgrade Scope Detail the specific components, configurations, and customisations that are in scope for the upgrade. Estimate The Upgrade Follow a structured process to create a realistic and data-driven estimate for the project. Resource The Upgrade Identify the roles, responsibilities, and skills needed to successfully deliver the upgrade. Define Testing Requirements Create a robust testing strategy to ensure a high-quality outcome and build confidence with stakeholders. Stage 3: Creating the Plan This final stage brings everything together into a formal plan. Page Description Produce Final Upgrade Plan Consolidate all the outputs from the previous steps into a single, actionable delivery plan.
Project Documentation to support you in upgrading Quantexa
One of Quantexa's priorities is to unlock the potential of your data while enabling customers to form their own self sufficient decision intelligence capability. Our Documentation team has outlined the requirements for upgrades, allowing you to prioritise needs against business benefit. This allows customers to assess through the following lenses: Which functionality update would help bring value to your organisation? Can additional functionality unlock other opportunities for Decision Intelligence in your organisation Are there any further technical or architectural requirements to consider when planning functional upgrades? How will these upgrades help with your end-user experience, or; Further the technical improvements of the platform? Where can you find this information? 📚️ As the Quantexa Releases are announced, the Documentation site explains the functional and non-functional updates, how they work, and how to implement the upgrades. As a Quantexa customer or partner, you will be able to access information around best practices to follow when upgrading your version of the Quantexa Platform on our Documentation site. In this section you will find guidance around Performing upgrades, Maintaining upgradable Quantexa implementations and Ongoing development during upgrades. Additionally, you can stay updated with the latest releases by subscribing to Release Announcements or reach out to your TAP (formerly CSMs & SSMs) to set up a meeting with the required team members. Quantexa can demonstrate the functional updates and discuss the benefits and effort requirements in order to help you plan out your platform roadmap, for your current needs or those you may have in the future. We are here to support you in any way you need. Additional resources For more information check out our blog: So you want to perform an upgrade? 💬 Do you have any questions or thoughts around scaling/upgrades that you would like to discuss with other Quantexa customers and experts? Ask your question in Quantexa Platform Support.How to Perform an Incremental Upgrade
You have completed your planning and preparation, and you are now ready to begin the hands-on execution of the upgrade. This guide provides the step-by-step technical process for performing a single "hop" of an incremental upgrade (e.g., migrating from v2.5 to v2.6). The fundamental principle of an incremental upgrade is to migrate your project one major version at a time, ensuring the codebase is stable and validated at each stage before proceeding to the next. The Key Tool: The Repository Tool The Quantexa Repository Tool will be the primary engine for your upgrade. It uses a file generator called PLOPjs and code-modification tools like Scalafix to automate a significant portion of the migration effort. While powerful, this tool does not cover every scenario, and manual changes may therefore be required. The Workflow for a Single Upgrade Hop For each hop in your upgrade roadmap (e.g., from v2.5 to v2.6), you will follow this four-phase process. Phase 1: Automated Migrations The first step is always to let the automated tooling do the heavy lifting. This is a highly scripted process. For detailed instructions on the commands, refer to the documentation on Running the repository tool. Configure the Tool: Locate the migration-config.json for the version hop you are performing. Update the projectPath to point to your repository. Run Scalafix Migrations: Execute the Scalafix migrations first. Run Plop Migrations: Execute the relevant Plop migrations. It is best practice to run these one by one to create a granular commit history. Pro-Tip: If your project uses split repositories (e.g., separate ETL and Apps repos), only run the migrations relevant to the repository you are currently working on. Phase 2: Manual Migrations With the automated changes applied, you will now execute the manual tasks from the backlog you created in the preparation phase. Your JIRA board is your guide for this phase. Execute Manual Migration Tickets: Work through the JIRA tickets you created for the mandatory manual migrations. Each ticket should contain the context and a link to the specific documentation needed for that single task. Address Customization Tickets: Work through the tickets related to your project's customizations. These tasks involve reviewing how the upgrade has impacted your custom code and applying the necessary fixes to make it compatible. Execute Optional Migration Tickets: If you decided to include any optional migrations in your scope, execute those tickets now. Pro-Tip: Make small, specific commits for each distinct manual change (e.g., "Fix custom scoring function for v2.6 API change"). This creates a clean, traceable history. Refer to project-example to see how the same migrations were applied in Quantexa's reference project. Phase 3: Compile and Validate With the automated and manual changes applied, this phase is focused on compiling the code and validating its integrity by running the automated test suite. Compile the Code: The first action is to run a full build of the repository. If any compilation errors arise from the applied migrations, resolve them. These are typically caused by API changes or updated method signatures that impact custom code. Validate with Unit Tests: Once the repository compiles successfully, run your full suite of automated unit tests. Address any tests that are failing as a result of the upgrade changes. A successful, clean build with all unit tests passing marks the end of the development work for this hop. Phase 4: Intermediate Validation (Optional but Recommended) Before moving to the next hop, it is highly recommended to perform some level of intermediate validation to catch issues early. Run your automated integration tests. Perform a small, local ETL run to ensure the core process works. If practical, deploy the applications locally to check for runtime errors. Repeat and Finalize Once you have completed this four-phase process for one hop, you repeat the entire workflow for the next hop in your upgrade roadmap (e.g., v2.6 -> v2.7). After the final hop is complete and you have a stable, building repository on your target version, the hop-by-hop development phase is over. You are now ready to proceed with the full Development Testing as outlined in your test plan.Quantexa Upgrade Knowledge Hub Release
Introducing the Quantexa Upgrade Knowledge Hub We are excited to announce the launch of the Quantexa Upgrade Knowledge Hub, a comprehensive collection of articles and best practices designed to make your platform upgrade process more transparent, predictable, and efficient. This new section on the Community site provides a step-by-step journey through every phase of an upgrade, from initial strategic planning through to final release. It's filled with practical advice, templates, and real-world considerations to help you succeed. Make sure you're signed in to the Community to access all the articles. What's Included in the Knowledge Hub? The Knowledge Hub is structured into clear, chronological sections that mirror the lifecycle of an upgrade project. Planning for Success This initial section covers all the strategic planning required before you begin. It guides you through defining your scope, choosing an upgrade approach, estimating effort, resourcing your team, and creating a robust testing strategy. The Preparation Stage Once your plan is in place, this section details the critical steps to prepare for execution. Learn how to set up tracking, gather your dependencies, establish a parallel development strategy, understand support channels, and turn your plan into actionable JIRA tickets. The Execution Stage This section provides the hands-on technical guidance for performing the upgrade itself. It covers how to execute an incremental upgrade hop-by-hop, practical considerations for testing, and a checklist for coordinating the final merge of your code. Version-Specific Guidance Dive into practical reviews and "what to watch out for" articles for specific Quantexa versions, including a detailed look at the 2.6 upgrade and the key migrations involved. Other Key Topics Explore important related subjects, such as the strategic benefits of migrating from DSL to QSL for graph scripting and how to reduce technical debt by removing Data Pack customizations using Fusion Extensibility. How Will This Knowledge Hub Help Your Team? This collection of resources is essential for anyone involved in a Quantexa upgrade: Upgrade & Delivery Leads can use it to structure their entire project plan and ensure no steps are missed. Developers & Data Engineers can follow the practical, hands-on guidance for technical migrations and testing. Architects & Product Owners can gain awareness of the strategic decisions, effort, and best practices involved in maintaining a current and healthy platform.2.6 Quantexa Upgrade Guide
Quick Upgrade Overview This article provides a practical review of the Quantexa 2.6 upgrade, highlighting key areas to watch out for based on collective project experience. It should be used as a strategic companion to the official technical guides. Official Migration Guide: 2.5→ 2.6 Upgrade Migration Guide Release Information: Community Release Announcement | 2.6.0 Release Notes The 2.6 upgrade has several non-negotiable prerequisites. Before beginning, it is essential to confirm that your project has already completed the following: All Data Sources on Data Fusion: If you have any data sources on the legacy Lenses framework, they must be migrated first. The full process is detailed in Migrating to Data Fusion. Scoring on Assess Framework: For projects still on Scoring Framework 1.0 (SF1), migrating to Assess is mandatory. This is not a direct technical conversion; it requires upfront design work to identify the right course for your project. The outcome could be to adopt modern Detection Packs if there is a good fit with your existing scoring logic, or it could be to re-implement your logic as custom scores within the Assess framework. It is strongly recommended to discuss your approach with a Quantexa Architect to determine the best path forward. For guidance on the technical migration, see Migrating from Scoring Framework 1.0 to Assess. Fusion-Compatible Data Packs: Ensure all Data Packs used by your project are the modern, Fusion versions. Check the Data Pack Compatibility Matrix to confirm your versions are compatible with Quantexa 2.6. Community Upgrade Guidance Core Product Changes Delta Lake Configuration: A straightforward configuration change in the spark-submit command. The key consideration is ensuring the Delta Lake version in your dependency-versions.gradle file is compatible with your Spark version, which can be verified on the Delta Lake release page. Explorer-*.json Configs: Simple manual edits are required to align with updated JSON schemas in some Batch Resolver configuration files. Batch Resolver Changes: A straightforward manual config change in reference.conf. The resolver now generates more output files; while a compatibility script is provided, it may be cleaner in the long run to update any downstream test code to handle the new format directly. Graph Script and Assess Imports: This is handled automatically by the repository tool, which refactors the library imports. Assess Changes: This area typically requires significant attention. The Batch Resolver data models have changed, which has a direct impact on Assess. If your scoring logic uses Entity Attributes, expect to refactor custom steps, as some fields have been removed or had their data types changed. Other Migrations Java 17 Upgrade: This is a significant undertaking that extends beyond the codebase. It requires upgrading the JDK across all environments: local developer machines, Docker images, CI/CD runners, and the production Spark clusters. It is crucial to engage with platform and infrastructure teams early to plan this. Gradle 8 Upgrade: The migration to Gradle 8.4 can be complex. An efficient approach is to generate a clean project using the 2.6 Repository Tool and use its working Gradle setup (build.gradle, settings.gradle, etc.) as a reference for your own project. LiteGraph to ScoringGraph Migration: A key migration with both automated and manual steps. This is a worthwhile effort, as ScoringGraph unlocks significant new functionality, most notably support for Entity-to-Entity edges and compatibility with the new Attribute types from the updated resolver configs. The repository tool handles much of the refactoring, but manual intervention is still required. Following the official Migrating to Scoring Graph guide closely is recommended. Alert Scorecard Migration: It is critical not to overlook this one-off data migration script for historical alert data, which is necessary to ensure re-alerting performs correctly once upgraded. Refer to Migrating from Alerting 2.5.x to 2.6.x for detailed guidance. Task View table Migration: This migration updates the database schema for the Task Data visible in the UI, typically by adding two new columns. The process depends on your database technology. For RDBMS systems, this involves running DDL migration queries. Crucially, these DDL scripts are not included in the main dependency bundle; they are provided in a separate ZIP file that must be downloaded from the Quantexa artifact repository. Additional Information For more general best practices on topics such as setting up development strategy, testing your migrated code, and releasing your upgrade, please refer to the other articles within the Upgrade Platform Library on the Quantexa Community site.Preparing For Success
You have successfully completed the planning phase and produced a comprehensive Upgrade Plan. You know what you need to do, how you're going to do it, and how long it will take. The Preparation Stage is the critical bridge between your plan and the hands-on execution of the upgrade. This is where you set up your project for success by establishing the right technical foundations, development processes, and support channels before the first line of code is migrated. A well-executed preparation phase minimizes friction, prevents common pitfalls, and ensures your development team can work efficiently and effectively once the upgrade work begins. Who is this guide for? This guide is primarily for Upgrade Leads and Developers who will be performing the hands-on work. Project Managers will also find the section on turning the plan into actionable tickets essential for tracking progress. What will you accomplish in this stage? By the end of this stage, you will have: Established a robust tracking system for all changes, decisions, and customizations addressed during the upgrade. Assembled all the necessary software, tools, and artifacts required for the upgrade. Established a clear development and environment strategy that allows upgrade work to proceed without halting business-as-usual development. Understood the support paths and processes available to you. Translated your high-level upgrade plan into a backlog of tangible, ready-to-work development tasks. The Preparation Process The following pages provide a step-by-step guide on how to best prepare for the execution of your Quantexa Platform upgrade. Page Description Implementing Upgrade Tracking: What It Is and Why It Matters Learn how to establish a robust system for tracking the upgrade process. This involves logging all issues encountered, the effort spent against tasks, and the details of customizations addressed or introduced. Maintaining these various trackers provides an essential audit trail and helps refine future upgrade estimates. Get Your Dependencies: What They Are and How to Obtain Them The essential next step. This guide covers how to locate and download all the required software versions, Quantexa release artifacts, and tools (like the Repository Tool) that you will need. Upgrading Without Stopping: Development Strategies and Environment Planning Learn how to set up your Git branching strategy (e.g., a long-lived upgrade branch) and plan your environment usage to isolate upgrade development from ongoing business as usual (BAU) work, ensuring both can proceed in parallel. Understanding Support During Upgrades: Where to Turn When Issues Arise Before you encounter a problem, it's vital to know where to go for help. This article outlines the different support channels available, from Documentation and Community to raising a formal Quantexa support ticket. Turn Your Upgrade Plan into Action: Creating JIRA Tickets The final preparation step. This guide shows you how to take the detailed "In Scope Tasks" list from your Upgrade Plan and convert it into a well-defined backlog of JIRA tickets, ready for your team to start working on.Estimate The Upgrade
With a clearly defined scope document in hand, you are now ready to build a reliable effort estimate. This process translates your detailed list of tasks into the person-days required to complete the project, forming the bedrock of your final upgrade plan. This guide provides a practical, bottom-up estimation process. It should be used alongside the official documentation's guide on Key factors impacting upgrade effort, which provides essential context on what makes an estimate go up or down. The estimation method you use will depend directly on the Upgrade Approach you selected earlier: Incremental or Reset. Estimating an Incremental Upgrade For an incremental upgrade, the most reliable estimate is built from the bottom up. You will estimate each individual line item from your "In Scope Tasks" table, which you created during the "Define Scope" phase. Action: Build your bottom-up estimate. For each task, estimate the development and unit testing effort. This detailed breakdown ensures you account for every specific migration and change required. To help with this process, detailed Work Breakdown Structure (WBS) templates for specific platform version upgrades can be requested from your Quantexa contact. Task / Migration Description Category Dev Effort (days) Unit Testing (days) Total (days) Upgrade Document Service response handling Manual Migration 3 1 4 Execute database schema update One-Off Task 0.5 0.5 1 Refactor deprecated custom-function-x Tech Debt 2 1 3 Run Repository Tool for v2.7 migration Automated Migration 1 2 3 ...add all other tasks... Sub-Total X Y Z Action: Validate your estimate with the top-down model. Now, use the best practice estimation approach from the Documentation Site as a sense-check. Start with the baseline (e.g., 8 days) and adjust it based on the key factors. Does your detailed, bottom-up total roughly align with the top-down adjusted baseline? If they are close, you can have high confidence in your number. If they are far apart, it's a crucial signal to review both your detailed estimates and your understanding of the influencing factors: Have you underestimated the complexity of a required migration? Have you overestimated the negative impact of your customizations? Action: Add contingency and formal testing. Finally, add a contingency buffer to your sub-total to account for unforeseen issues. Remember, this estimate is for the development and unit testing effort only. The effort for formal testing cycles (SIT, UAT, OAT) must be estimated and added separately. Estimating a Reset A "Reset" project is estimated in much the same way as a brand-new Quantexa implementation. The focus is on estimating the effort required to selectively re-implement the desired functionality in a new, clean repository. Action: Estimate the "re-implementation" effort for each functional block. For each major component of your current solution (e.g., Entity Resolution, Scoring), estimate the effort to build it in the new target version. When doing so, you must factor in the effort to align with modern best practices and to remove the technical debt you identified in your existing customizations. Functional Block Re-Implementation Effort (days) Notes Data Ingestion & Parsing 15 Includes refactoring 3 custom parsers. Entity Resolution 20 Involves re-implementing logic to align with new best practices. Scoring & Alerting 10 UI & Document Views 12 Includes rebuilding 2 custom document viewers. ...add other functional blocks... Sub-Total X As with the incremental approach, a healthy contingency should be added to this sub-total, and formal testing should be estimated as a separate line item. ⚠️Warning: Avoid the "Big Bang" Trap A reset upgrade provides a tempting opportunity to tackle other major platform migrations at the same time (e.g., historical Fusion or Assess migrations, Parsers v4). While this can be efficient, be extremely careful about letting the scope balloon. The effort to validate major functional changes, combined with the cost of a very long-running project, can quickly become unmanageable. It is often better to treat these as separate, subsequent projects.2.9 Quantexa Upgrade Guide
Table of Contents Quick Upgrade Overview Community Upgrade Guidance Dependencies and Platform Modernisation User Interface Changes Pre-built Images and Pre-packaged Artifacts Backend and Data Pipeline Changes Data Packs Migration Recent Deprecations Software Compatibility Additional Information Quick Upgrade Overview The 2.9 Quantexa Upgrade consists of five main parts: Dependencies and Platform Modernisation User Interface (UI) Changes Pre-built Images and Pre-packaged Artifacts Backend and Data Pipeline Changes Data Packs Migration The 2.9 release is one of the most structurally significant Quantexa upgrades to date. The headline change is the move to pre-built Docker images for App Tier applications. Rather than compiling application modules project-side, projects now use the Image builder plugin to layer project-specific extensions onto Quantexa-provided base images. This fundamentally changes the build and deployment model and drives many of the API refactoring changes across the platform. On the dependencies side, this release involves major framework upgrades: Gradle 9, Spring Boot 4, Node.js 24, and Angular 21. Elasticsearch 7 and OpenSearch 1, deprecated in 2.8, have been fully removed - projects must be on Elasticsearch 8/9 or OpenSearch 2. Elasticsearch 9 is newly supported. While the Repository Tool handles most of the automated migration, projects with custom Gradle build code, custom Spring configuration, or custom UI code should budget additional manual effort. The User Interface tier includes many configuration restructuring changes. Global UI configuration, Explorer dynamic configuration, and Search 2 configuration all move to versioned directory-based formats. The Home Page reaches General Availability (GA), several Investigation modules are removed, and PrimeNG is replaced with a Quantexa-maintained fork. Data Packs 2.3.x remain compatible with Quantexa 2.9 and require Parsers 4.2.3 or later. This page provides additional guidance for the 2.9 Quantexa Upgrade. For the full list of required migration steps, please refer to the Documentation site migration guide: 2.8 → 2.9 Upgrade Migration Guide. Release Notes: Community Release Announcement 2.9 Release Notes Community Upgrade Guidance Dependencies and Platform Modernisation Gradle 9 Upgrade This is one of the most impactful changes in the 2.9 upgrade. Gradle 9 removes several build script APIs and tightens behavioural rules that previously generated only deprecation warnings. The Repository Tool automates the migration for patterns used in Quantexa-generated build files, covering changes such as replacing configurations.all with configurations.configureEach, updating archivesBaseName to archiveBaseName.set(), and reordering sourceSets blocks. However, deployments with custom Gradle build code will almost certainly require additional manual changes. After the automated migration, run ./gradlew build and resolve any remaining failures. Common follow-up tasks include replacing usage of removed org.gradle.util.CollectionUtils methods and reviewing any include directives that the migration commented out in settings.gradle. Spring Boot 4 Upgrade The mid-tier has been upgraded from Spring Boot 3.x to Spring Boot 4.0, introducing breaking changes for all deployments. A Scalafix rule handles Scala source file migrations, and a Plop migration covers common changes such as relocating configuration keys (server.error → spring.web.error) and updating Spring Boot imports. However, the Plop migration cannot cover every Spring Boot 4 scenarios, especially for custom code. After running the automated migrations, manually review the official Spring Boot 4.0 Migration Guide for any additional changes your deployment requires. Note that moving to pre-built images also eliminates many Spring bean extension points, replacing them with Scala singleton objects (see Pre-built Images section below). Elasticsearch 7 and OpenSearch 1 Removed Following their deprecation in 2.8, Elasticsearch 7 and OpenSearch 1 have been fully removed. Deployments still on ES7 must upgrade to Elasticsearch 8 or 9 before upgrading to 2.9. Deployments on OpenSearch 1 must upgrade to OpenSearch 2. Elasticsearch 9 is now supported alongside Elasticsearch 8. Quantexa recommends upgrading to ES9 for the latest performance and security improvements. However, note the following limitations in 2.9.0: EmbeddedElasticServer is not supported for ES9 - deployments using it for local testing must migrate to Docker-based Elasticsearch Offline Indexing is currently not supported with ES9 OpenSearch 3 is not supported in this release The elastic*-builder NPM packages used by the UI for ES7 query building have also been removed. Node.js 24 and Angular 21 Node.js has been updated from version 22 to 24 (required by Angular 21). Angular has been updated to version 21. Plop automatically handles the UI code migration. As with the 2.8 Angular upgrade, Plop may fail to migrate package-lock.json to a consistent state - if you encounter UI build errors, delete node_modules and package-lock.json from the web-app folder and run npm install to regenerate them. Ensure all CI/CD pipelines, development environments, and Docker build stages reference the new Node.js version. User Interface Changes Home Page (General Availability) The Home Page reaches General Availability in 2.9, providing a configurable landing page with widgets for quick access to recent activity, tasks, and investigations. A Plop migration is available for initial setup. Projects should review their Home Page configuration and test the landing experience post-migration. Investigation Modules Removed The Investigation modules deprecated in 2.8 (replaced by Resource Viewer) have been fully removed. The affected modules include CompoundGraphViewModule, SourceViewerPluginModule, RecordViewerPluginModule, and others. A Plop migration handles the removal. Projects should have already migrated to Resource Viewer during the 2.8 upgrade. Additionally, the Custom Bulk Search component has been removed - Bulk Search configuration is now fully managed through the standard configuration approach. Global UI Configuration Migrated to Versioned HOCON The Global UI configuration has been restructured from a monolithic JSON file to a versioned directory containing HOCON configuration and a version.conf file declaring the schema version. This enables automatic schema migration during platform upgrades. Plop creates a config/global-ui/ directory structure and updates application.yml paths. Several previously unused fields (validFrom, validTo) have also been removed. Projects must verify that deployment manifests (Helm Chart or QAT) are updated to mount the new directory as a ConfigMap volume. Explorer and Search 2 Configuration Split Explorer dynamic configuration has been split from a single monolithic file into per-schema .conf files within an explorer-dynamic-configuration/ directory, with an index.conf file. Search 2 configuration follows the same versioned directory pattern, migrating from a single JSON file to config/search2/ containing search2-dynamic-config.conf and version.conf. Both Plop migrations handle the conversion, update application.yml paths, and add ConfigMap entries to deployment scripts. Projects must verify that the updated volume mounts are present in their deployment manifests. PrimeNG Fork and Dependency Cleanup PrimeNG has been replaced with @quantexa/primeng-fork, a Quantexa-maintained fork. All primeng/* imports in TypeScript, CSS, and SCSS files must be updated - a Plop migration handles this automatically. Additionally, Web Core peer dependencies have been removed from project-level package.json and are now managed by the @quantexa/web-core package. After migration, regenerate package-lock.json by deleting node_modules and the lock file, then running npm install. To keep dependency overrides up to date between major releases, use the src:cve-update script. The project-level ESLint configuration and its dependencies have also been removed. Projects that require ESLint must configure it independently. UI script and import paths have been updated: the lint, src:migrate-scss, and eslint-config:link scripts are removed, and @quantexa/script-lib imports must be updated to @quantexa/utils and @quantexa/build-tools. Alternative Network Layouts (General Availability) Alternative Network Layouts, which provide different ways to visualise entity networks, reach General Availability in 2.9. A Plop migration renames edge configuration properties. Projects using custom network layout configuration should review the updated property names. Localisation Configuration Restructured The localisation configuration has been restructured for multi-language support. This is a breaking change with Plop migration support. Projects should verify their localisation setup post-migration. Low-Code Configuration (LCC) Validation Tests LCC validation tests have been migrated to a new location. A Plop migration updates test locations and imports. Continue to run configuration validation tests early in the upgrade process to catch configuration errors before deployment. Pre-built Images and Pre-packaged Artifacts Application Pre-built Docker Images This is the most architecturally significant change in 2.9. Applications are no longer compiled project-side. Instead, Quantexa provides pre-built Docker images for App Tier services (all applications except the UI and app-fusion). Projects use the Image builder plugin to layer project-specific extensions onto these base images. The migration involves seven steps, which must be completed in order: Migrate application extensions to dedicated modules (Manual) - custom code in application modules (e.g. app-security/src/main/, app-investigate/src/main/) must be moved to dedicated extension modules with recognized naming conventions that the Image builder plugin auto-discovers. Register Foreign Documents for Fusion data configuration (Manual) - applications now read data configuration from fusion-data-config.json rather than compiled Scala models. Foreign Documents not automatically included must be registered via .qdocuments and .qmodel files. Migrate SAML metadata files (Plop) - the classpath: protocol for SAML metadata is no longer supported; XML files must be moved from app-security/src/main/resources/ to a mounted config/saml/ directory. Migrate optional modules to feature flags (Plop) - optional service dependencies in build files are replaced with feature flags in deployment configuration. Add Image builder plugin configuration (Plop) - adds the plugin to the root build and creates the applications.gradle file. Migrate each application to pre-built images (Plop) - replaces each application module with a .gitkeep, updates settings/libraries/applications Gradle files. Update deployment configuration (Manual) - applications with project-specific extensions are renamed with a -project suffix (e.g. app-resolve becomes app-resolve-project). Deployment configuration must reference the new names. This migration has a cascading impact across the platform. Many of the API changes in Visualization and Exploration (see below) are directly driven by this change - extension points that previously relied on Spring bean registration in compiled application modules must now use Scala singleton objects in dedicated extension modules. Projects should follow the Application Pre-built Image Migration Guide carefully. For projects that need help adopting container-based deployment, speak to your Quantexa Architect. They can help assess your current deployment model and plan the transition to pre-built images. Impact on Local Development The move to pre-built Docker images changes the local development workflow. Previously, developers could run mid-tier applications locally using bootRun or java -jar commands. With pre-built images, these applications are no longer compiled project-side, so the traditional local run scripts no longer apply for most App Tier services (the UI and app-fusion are still built project-side). There are three options for development workflows going forward: Cloud-based or development environment development (recommended) - develop against a shared or personal Kubernetes development namespace with CI/CD pipelines configured for rapid iteration. This is the approach Quantexa uses internally and is the recommended path. Projects should work with their Quantexa Architect to set up development namespaces that support concurrent developer activity, including sizing the environment to handle multiple developers working in parallel. Local container-based development - use Minikube, Docker Desktop, or an equivalent local Kubernetes environment to run pre-built images on a developer laptop. This preserves the ability to work locally but requires container tooling to be available on the development machine. The overhead of Docker itself is relatively low, but the growing footprint of the full application stack may be challenging on lower-specification machines. Projects should verify that laptop or VDI specifications are sufficient for this approach. Pre-built boot JARs (fallback) - pre-built Spring Boot JARs are available for projects that cannot adopt container-based development locally. However, this means local development uses a different deployment pattern to higher environments, requiring the project to maintain two deployment approaches and accept the risk of environment divergence. The same structural migration steps (dedicated extension modules, Image builder plugin configuration) still apply to ensure compatibility with repository migrations in later Quantexa versions. Projects upgrading to 2.9 should factor the development workflow transition into their migration effort estimates. For environments with strict change management requirements where cloud-based development namespaces are not readily available, discuss options with your Quantexa Architect early in the upgrade planning process. Pre-packaged Batch Shadow JARs Quantexa now provides pre-packaged shadow JARs for Batch tier tasks, eliminating the need to compile batch platform code from source. A Plop migration updates project structure and build configuration. Projects with custom batch tasks should review the migration to ensure their custom tasks integrate correctly. Backend and Data Pipeline Changes Data Fusion and ETL Several changes affect the Data Fusion and ETL pipeline: ENGFull renamed to BatchFull (Plop) - generated ETL script classes have been renamed from ENGFull{Model}ETLScript to BatchFull{Model}ETLScript. Template files have similarly changed from runAllENG.sh to runAllBatch.sh. Metadata excluded from hash ID calculation (Plop) - the metaData field is now excluded from hash ID computation by default when using hashIdField configuration. This improves Entity Resolution (ER) stability during incremental ETL runs. To preserve the previous behaviour, add metaData to the including list in your hash ID configuration. fusion-spark-utils removed - projects using this library must migrate to the replacement APIs. Document type validation against Fusion config (Manual) - document type definitions in Resolver configuration are now validated against Fusion configuration at startup. If a Document type is defined in Resolver but missing from the corresponding .qdocuments file, the application will fail to start with an IllegalStateException. Resolver and Batch Resolver Several Resolver APIs have been cleaned up: liftAttributes removed (Manual) - the liftAttributes configuration option has been removed without replacement. This changes how Record Attributes with mixed types are handled across Document types. Projects should check Batch Resolver logs for Record attributes to lift: entries to identify impact. Path methods removed (Manual) - several Path convenience methods have been removed from the Batch Resolver API. Resolver configuration cleanup (Plop) - several deprecated fields have been removed: dataModelRootClass, dataModelRecordExtractor, intelligenceModelClass, intelligenceModelExtractor from document/intelligence definitions; icon from entity definitions; availableIcons from UDI document definitions. Entity Store Key changes to the Entity Store: Custom External Attributes (Manual) - must now be defined in a dedicated custom-external-attributes module and configured via the external-attributes.calculations-object property. Previously, these could be defined in any module and injected as a Spring Bean. Custom Attribute Functions: ValidatedNel → Either (Manual) - the return type has changed from ValidatedNel to Either[List[String], T], and the expectedFields and name variables have been removed from the relevant traits. Custom Data Quality Functions (Manual) - the EntityQualityCheckAggregation interface now requires usedEntityAttributes and usedRecordAttributes properties. Error handling changes (Manual) - the Entity Store REST API now returns clearer error responses (validation errors return 400, timeouts return 503). Deployments with custom error handling logic must update their implementation. Change Log metadataPath (Plop) - Entity Store Change Log, CRUD Log, Document Log, and Batch ETL to Stream scripts now require a metadataPath configuration property. Scoring The elastic*-builder Gradle dependency has been removed. The Scoring Dynamic subproject requires migration to a new structure; follow the Scoring Dynamic Subproject Migration Guide. Both have Plop support, but deployments with custom Scoring configuration should review changes carefully. Data Integration Several Data Integration changes: Kafka Ingest error handling configuration (Manual) - error handling configuration has been refactored as part of the pre-built image migration, moving to a dedicated extension module. RecordExtractionModelConfig refactored (Manual)-RecordExtractionModelConfig has been replaced by RecordExtractionModelConfigBuilder. Search Related Entities configuration (Manual) - configuration has been updated as part of the pre-built image migration. Combined Ingest service removed - the combined ingest service has been fully removed. DatasourceTestSpec deprecated - DatasourceTestSpec is deprecated in favour of ETLTestSpec. Additionally, Elasticsearch-backed data source integration tests (GenericFullEtlRun ES variants) have been removed. Q AI - Q Assist Q Assist Workspace Agents reach Public Preview in 2.9. LLM configuration for Q Assist has been updated - projects using Q Assist must review and update their LLM provider configuration. This is a manual change. Decision Systems Decision Systems 2.9.0 includes breaking changes. Key changes include: initialTaskGraph is now required, Hub and Spoke Scores have been removed, and Contextual Data Mapping has been converted to Monitored Data. See the 2.8 → 2.9 Decision Systems Migration Guide for step-by-step guidance. Data Packs Migration Data Packs 2.3.x are compatible with Quantexa 2.9.x and require Parsers 4.2.3+, 4.3.x, or 4.4.x. Parsers 3 has not been supported since Quantexa 2.8. If your project has not yet completed the Parsers 4 migration, complete it before upgrading. For planning and step-by-step guidance, see: Why & How to Plan a Parsers 4 Migration Step-by-step Guide to Migrate to Parsers 4.3 There are no Data Packs-specific breaking changes in the 2.9 upgrade beyond what is covered by the platform migration. Projects already on Data Packs 2.3.x with Parsers 4 can continue using the same version. For the full Data Packs compatibility matrix, see Data Packs Compatibility Matrix. Recent Deprecations PyQSS Import Path The PyQSS import path has been deprecated. Projects using PyQSS should update their imports to the new path. I/O Handler Environment Variables I/O handler environment variables have been deprecated in favour of updated configuration. smart_open Library The smart_open library has been deprecated. icon Property in Resolver and Fusion Document Definitions The icon property in Resolver and Fusion Document definitions is deprecated. While the property still functions, it will be removed in a future release. DistinctValue for List Type Fields Using DistinctValue for List type fields is deprecated. DatasourceTestSpec The DatasourceTestSpec class is deprecated in favour of ETLTestSpec. Bundled libpostal Bundled libpostal in app-kafka-record-extraction and app-search-related-entities images is deprecated. Projects relying on the bundled version should plan to provide their own libpostal installation. transliterateElements The transliterateElements configuration has been deprecated. Software Compatibility Check software compatibility. Key changes for 2.9: Apache Spark: 3.5.x Java JDK (Application Tier): 17 Node.js: 24.15.0 (upgraded from 22) npm: 11.12.1 (upgraded from 10.9.0) Gradle: 9.3.1+ (upgraded from 8.4+) Elasticsearch: 8 and 9 (ES7 removed); OpenSearch 2 (OS1 removed) Redis / Valkey: Redis 6.x, 7.x, 8.x and Valkey 7.x, 8.x, 9.x (now mandatory) Kafka: 3.x, 4.x CPython: 3.11.x, 3.12.x (3.10.x removed) Scala: 2.12 MySQL: 8.x, 9.x PostgreSQL: 13.x to 18.x Additional Information Useful resources: Upgrade Best Practice for instructions before commencing an upgrade. Ongoing development during upgrades for best practice guidance on continuing with meaningful development during an upgrade. Quantexa Upgrade Knowledge Hub for comprehensive guidance on planning and executing upgrades. Application Pre-built Image Migration Guide for the step-by-step guide to migrating to pre-built Docker images. Scoring Dynamic Subproject Migration Guide Angular 21 Migration Guide Upgrading from Elasticsearch 8 to Elasticsearch 9 2.8 → 2.9 Decision Systems Migration Guide 2.9 Release Note Walkthrough video walkthrough of the 2.9 release. Parsers 4 Migration Guidance for planning and executing the Parsers 4 migration. Data Packs Compatibility Matrix Follow the Release Announcements Topic to receive notifications of releases. Quantexa Release Notes for changes and information specific to different versions of Quantexa.Define Your Upgrade Approach
Having created your Upgrade Roadmap, the next critical decision is to define how you will execute the work. There are two fundamentally different strategies for upgrading your Quantexa platform: the Incremental Upgrade and the Reset Upgrade. This page will guide you in selecting the right approach for your specific context, considering factors like your roadmap's complexity, your solution's health, and your team's capacity. At a Glance: Comparing the Approaches Approach Key Characteristic Best For... Primary Benefit Key Consideration Incremental Upgrading through each major version sequentially ("hopping"). Solutions that are well-aligned with Quantexa best practices and are only a few versions behind. Efficiency. Maximises automation using the Repository Tool. The Repository Tool is less effective on solutions with high technical debt. Reset Creating a new, clean repository on the target version and migrating code selectively. Solutions that are many versions behind or contain significant technical debt. Clean Slate. An opportunity to remove technical debt and realign to best practices. Higher initial manual effort to set up and selectively migrate logic. In-Depth: The Incremental Upgrade This is the most common and efficient approach. It involves upgrading your existing repository through each major version between your current state and your target destination. For example, moving from v2.5 to v2.8 would involve separate, sequential upgrades to v2.6, v2.7, and finally v2.8. This approach makes maximum use of Quantexa's Repository Tool, which is designed to automate much of the migration effort between sequential versions. This is the default and recommended option if you are only upgrading a single major version (e.g., from 2.7 to 2.8). When performing an incremental upgrade across multiple versions, you have two release strategy options: Option 1: Single Go-Live Perform each version "hop" in quick succession within your development environment, but combine them into a single release package. This package goes through one cycle of SIT, UAT, and Production deployment. Benefits: Reduced Testing Overhead: One round of formal SIT/UAT saves significant time and effort. Faster Time-to-Target: Achieves the final goal of deploying the target version in the shortest overall project duration. Developer Efficiency: Developers build deep expertise by performing the "hops" back-to-back, increasing speed. Option 2: Staged Go-Live (Release per Hop) Treat each version "hop" as a mini-project. You develop, test (SIT/UAT), and release each incremental version to Production before starting the next hop. Benefits: Lower Risk per Release: Each deployment is smaller and more contained, simplifying testing and root cause analysis if issues arise. Incremental Value: End-users can benefit from new features and improvements more regularly. Increased Flexibility: Allows your team to address other business priorities between upgrade stages. In-Depth: The Reset Upgrade This approach involves creating a brand-new, empty repository on the target platform version. Your team then selectively migrates or completely rebuilds the required logic (data interfaces, Entity Resolution, Scoring, etc.) from the old repository into the new one. This is a strategic choice to "pay down" technical debt. Benefits: Your implementation is multiple major versions behind the latest Quantexa Platform release. The current solution contains a high degree of technical debt, making an incremental upgrade complex and risky. You need to make significant architectural changes or re-align to modern Quantexa best practices. While this approach requires more upfront manual effort, it ensures that your repository is clean, modern, and optimized for future scalability and smoother upgrades. How to Choose Your Approach Your choice will depend on the specifics of your Quantexa deployment and the goals of your upgrade. If your solution is well-aligned with best practices and your Upgrade Roadmap involves only one or two hops, the Incremental approach is almost always the best choice. If your solution suffers from significant technical debt or your Upgrade Roadmap spans many major versions, the Reset Upgrade approach offers a powerful opportunity to reset for the future, even if the initial effort is higher. Quantexa's Divergence Tool can help provide insights into how far away from best practice the deployment is in certain areas The output of this step is a clear, documented decision on the approach you will take. Action: State your chosen upgrade approach. Example: We will follow an Incremental Upgrade approach with a Single Go-Live release strategy.
Read now: What it's Like to Perform an Upgrade
Read all about our experience and key takeaways when upgrading a repository from version 2.1 to 2.3. The upgrade was performed and released in early 2023 shortly after version 2.3 became available. The main motivation behind this particular upgrade was to upgrade the batch tier software including versions of EMR & Spark which Quantexa 2.3 supported. The flexibility of the new Data Viewer and other newly released features were also important value-adds. What's it Like to Perform an Upgrade? - Quantexa Community This blog describes the experience and key takeaways when upgrading a repository from version 2.1 to 2.3. The upgrade was performed and released in early 2023 shortly after version 2.3 became available. Why upgrade Quantexa versions? As with any software, the advantages of upgrading to newer versions include: Access to new…
