2.8 Quantexa Upgrade Guide
Table of Contents Quick Upgrade Overview Community Upgrade Guidance Dependencies and Platform Modernisation User Interface Changes Batch Resolver and Entity Resolution Redis / Valkey Distributed Caching Data Packs Migration Recent Deprecations Software Compatibility Additional Information Quick Upgrade Overview The 2.8 Quantexa Upgrade consists of five main parts: Dependencies and Platform Modernisation User Interface (UI) Changes Batch Resolver and Entity Resolution (ER) Redis / Valkey Distributed Caching Data Packs Migration The majority of the dependency changes (Spark 3.5 migration, Cats Effect 3, Apache Pekko, library version bumps) are handled by automated Plop and Scalafix migrations via the Repository Tool. However, any custom code that deviates from best-practice patterns, particularly Cats Effect or Akka usage, will require manual migration. A new Quantexa license is also required before starting. The User Interface tier sees the most user-facing change. Resource Viewer reaches General Availability (GA) and replaces several Investigation modules, Transaction Viewer has been fully removed (migrated to Data Viewer), Angular has been updated to version 18, and the web-app folder structure has been simplified. Projects should plan to test the UI thoroughly post-migration. Batch Resolver receives several important changes: Network Generation and time-splitting have been removed, Classic mode is deprecated, and the Entity Attributes output model has changed. Filtered Compounds now use QSL instead of QEL. Most of these have Plop or Scalafix support, but projects with custom Batch Resolver scripts should budget additional effort. Redis / Valkey distributed caching is introduced as optional in 2.8 but becomes mandatory from 2.9 onwards. Projects are strongly encouraged to set up Redis during the 2.8 upgrade to avoid a separate infrastructure change later. Data Packs 2.3.x are compatible with Quantexa 2.8 and require Parsers 4.2.3 or later. Projects still on Parsers 3 must complete the Parsers 4 migration before upgrading to 2.8. This page aims to provide additional guidance related to the 2.8 Quantexa Upgrade. For the full list of required migration steps, please refer to the Documentation site migration guide: 2.7 → 2.8 Upgrade Migration Guide. Release Notes: Community Release Announcement 2.8 Release Notes Community Upgrade Guidance Dependencies and Platform Modernisation New Quantexa License Required A security improvement to the Quantexa license means that existing licenses are not compatible with 2.8.0. You must request a new license before deploying the upgraded platform. This is a prerequisite that should be actioned early in the upgrade process to avoid blocking deployment. Spark 3.5 Migration Support for Spark 3.3 and 3.4, deprecated in 2.7, has been fully removed. Projects must now run on Spark 3.5. The BOM changes are applied automatically by Plop. The primary manual effort comes from ensuring custom project-owned Spark scripts are compatible with Spark 3.5 and upgrading Spark installations across all environments (local, CI/CD, and production). During internal migrations, no significant behaviour changes were found, so this should be a low-risk change for most projects. Cats Effect 3 This is one of the more impactful dependency upgrades. The cats-effect, fs2, and fs2-Kafka libraries have all been updated to version 3.x. Code following the best-practice structure in Task Loading, Scoring, Alerting, Entity Store, and Data Source Integration Tests is migrated automatically. However, any custom code built upon earlier versions of these libraries, such as custom Kafka applications, must be manually migrated following the official Cats Effect 3 migration guide. Projects should use Project Example as a reference. Akka Replaced by Apache Pekko Due to CVEs raised against the Akka library, Quantexa has switched to Apache Pekko. This only affects deployments using app-graph-script. The migration is straightforward: ensure import akka.util.Timeout statements are changed to org.apache.pekko.util.Timeout. A Scalafix migration is available. Other Library Upgrades Several other libraries have been updated, all with Plop or Scalafix support: Scalatest upgraded from 3.1.4 to 3.2.19 ScalaCheck upgraded (artifact name changed) Chimney version updated Java Faker replaced by Datafaker ETL Util dependencies renamed (batch-common-api → spark-api, common-spark-utils → document-index-input-reader) spark-excel artifact and package changed spoiwo package updated from 1.x to 2.x org.apache.commons.io must now be shaded in the Batch tier JSON Configuration Validation A notable change to be aware of is that configuration validation on application startup now causes the application to fail if Perspective, Bulk Search, or Foreign Document JSON configuration files contain unknown fields. Previously, these invalid fields were silently ignored. This means that configuration errors that have existed undetected in your project may now surface as startup failures. It is recommended to run the Configuration validation tests early in the upgrade to identify and fix any issues before attempting to deploy. User Interface Changes Resource Viewer (General Availability) Resource Viewer is the most significant user-facing change in 2.8. It provides a unified, tabbed interface for viewing and interacting with Documents, Entities, and other resources. All projects upgrading to 2.8 will inherit the Resource Viewer, which replaces several existing Investigation modules including CompoundGraphViewModule, SourceViewerPluginModule, RecordViewerPluginModule, and others. These replaced modules are now deprecated and will be removed in the next major release. This introduces breaking changes for deployments using Low-Code Configuration (LCC) or Record Viewer. Projects should follow the Resource Viewer Migration Guide and plan for testing time to verify the UI behaves as expected, particularly around any custom UI configuration. Transaction Viewer Removal Following its deprecation in 2.4, Transaction Viewer has been fully removed. Data Viewer is the recommended replacement, offering feature parity plus improvements such as no record limit, column-level UI filtering, and Scoring plugin integration. Projects still using Transaction Viewer must complete this migration. Follow the Transaction Viewer Migration Guide for guidance. Angular 18 and web-app Simplification The UI has been updated to Angular 18. Plop automatically handles the migration of your deployment's UI code. However, due to the many changes to NPM dependencies, Plop may fail to migrate web-app/package-lock.json to a consistent state. If you encounter UI build errors after migration, delete the node_modules and package-lock.json files from the web-app folder and run npm install to regenerate them. The web-app folder structure has also been simplified, removing boilerplate files and simplifying how the application is bootstrapped. Tab configuration previously in MainViewComponent has moved to a dedicated tab-config.ts file. Global UI Configuration Changes The file structure used for global settings in Low-Code Configuration has changed. If your deployment uses global configuration loaded through JSON files on deployment, you must migrate to the new format. Follow the Global UI Configuration Migration Guide. Batch Resolver and Entity Resolution Network Generation and Time-splitting Removed The Network build and Create ScoringGraph stages of Batch Resolver, which produce Natural Networks, have been removed following their deprecation in 2.7. Projects still using these must migrate to Graph Scripting QSL for network generation. The associated configuration options (appendFilteredDocuments, createNetworkStats, scoringGraphMaxNodeCount, etc.) must be removed from Batch Resolver configuration files. Similarly, the time-splitting functionality has been removed. All timeSplit fields must be removed from Resolver JSON configuration. Filtered Compounds: QEL to QSL Migration The Quantexa Expression Language (QEL) expression field within Filtered Compound definitions has been replaced with a QSL where field. A Plop migration is available to translate existing QEL filters to QSL. Projects should review the translated filters to ensure correctness, particularly for complex expressions. Additionally, Batch Resolver now warns if Filtered Compound definitions reference Record Attributes not defined in the input data. These warnings will be upgraded to errors in a future release, so projects should address them during the upgrade. Note that the default Address Resolution Template in the Data Packs core fragment contains an erroneous filteredCompoundDefinitions block referencing a rootCountyCode attribute; if present, this block should be removed. Entity Attributes Output Model Changes The excluded, subEntities, and linkExcluded fields have been removed from the Batch Resolver Entity Attributes output. A new failedDataQualityChecks field has been added. A Scalafix migration is available, but due to Scalafix limitations, imports referring to the removed SubEntity model must be removed manually. Classic Mode Deprecated Batch Resolver Classic mode is deprecated and will be removed in a future release. Projects should migrate to Resolver mode. To replicate previous Classic mode exclusions, use Element exclusions in ETL. See Migrating from Classic to Resolver mode for guidance. Calling Batch Resolver and Graph Scripting QSL from Code Deprecated Calling Batch Resolver and Graph Scripting QSL stages from code is deprecated and will be removed in a future major release. Projects must migrate to using the pre-defined scripts under com.quantexa.batchresolver.scripts and com.quantexa.graphqsl.scripts directly with spark-submit. If you previously used custom scripts to insert Run IDs from Metadata into file paths, use the updated dataPaths configuration. Redis / Valkey Distributed Caching Quantexa 2.8 introduces distributed caching with Redis or Valkey. While Redis is optional in 2.8, it becomes mandatory from 2.9 onwards. We strongly recommend configuring Redis during the 2.8 upgrade so that this infrastructure is in place before upgrading to 2.9. The Repository Tool automatically adds a baseline application-redis-profiles.yml configuration file and updates docker-entrypoint scripts. Redis can be deployed via the Quantexa Helm Chart (Redis Sentinel), Google MemoryStore for QCP-hosted projects, or a manually deployed Redis/Valkey instance. To enable Redis caching, set the following feature flag in your Spring configuration: quantexa.caching.redis-caching.enabled: true For full deployment guidance, see the 2.7 → 2.8 Migration Guide, section "Distributed caching using Redis or Valkey". Data Packs Migration Data Packs 2.3.x are compatible with Quantexa 2.8.x and require Parsers 4.2.3 or later. Parsers 3 is no longer supported from Quantexa 2.8 onwards. Projects still on Parsers 3 must complete the migration to Parsers 4 before upgrading to 2.8. This migration should be treated as a separate workstream and is not recommended to be performed alongside the 2.8 platform upgrade. For planning and step-by-step guidance, see: Why & How to Plan a Parsers 4 Migration Step-by-step Guide to Migrate to Parsers 4.3 For the full Data Packs compatibility matrix, see Data Packs Compatibility Matrix. Recent Deprecations Batch Resolver Classic Mode Classic mode is deprecated and will be removed in a future major release. Projects must migrate to Resolver mode. See Migrating from Classic to Resolver mode. Elasticsearch 7 and OpenSearch 1 Elasticsearch 7 and OpenSearch 1 are deprecated in 2.8 and will be removed in 2.9. Projects should plan to upgrade to Elasticsearch 8 or OpenSearch 2. Search 1 Search 1 is deprecated. Projects should migrate to Search 2. Connect Connect is deprecated in 2.8. Quantexa Feature Engineering Quantexa Feature Engineering is deprecated. Projects using version 0.9.0 must replace their imports with their own Spark transforms. Graph Scripting DSL Following its deprecation in 2.7, the DSL Conversion to Scoring Graph methods have been removed in 2.8. The Entity Statistical Profile Testing Framework's use of DSL models is also deprecated - projects must migrate to using the CalculatedEntityAttributes Batch Resolver output. Parsers 3.x As noted above, Parsers 3.x is no longer compatible with Quantexa 2.8. Projects should complete migration to Parsers 4.x before upgrading. Software Compatibility Check out software compatibility. Key changes for 2.8: Apache Spark: 3.5.x only (Spark 3.3 and 3.4 removed) Java JDK (Application Tier): 17 Node.js: 22.11.0 npm: 10.9.0 Gradle: 8.4+ Redis / Valkey: 6.x, 7.x (optional in 2.8, mandatory from 2.9) Elasticsearch: 7 (EOL) and 8; OpenSearch 1 and 2 Kafka: 3.x, 4.x CPython: 3.10.x, 3.11.x MySQL: 8.x, 9.x PostgreSQL: 13.x to 18.x Additional Information Useful resources: Upgrade Best Practice for instructions before commencing an upgrade. Ongoing development during upgrades for best practice guidance on continuing with meaningful development during an upgrade. Quantexa Upgrade Knowledge Hub for comprehensive guidance on planning and executing upgrades. Parsers 4 Migration Guidance for planning and executing the Parsers 4 migration (required for 2.8). Resource Viewer Migration Guide Global UI Configuration Migration Guide Entity Store Load External Attributes Migration Migrating Elasticsearch Clients for Scoring Follow the Release Announcements Topic to receive notifications of releases. Quantexa Release Notes for changes and information specific to different versions of Quantexa.How to Release Your Upgrade: Approaches and Considerations
You have successfully tested your upgrade and are now dev-complete. The final stage of the execution process is to plan and execute the merge of your upgrade branch back into the main codebase, making it the new standard for all future development. A smooth merge is the result of careful coordination and communication. This guide provides a step-by-step checklist to ensure your upgrade is integrated without disruption. Step 1: The Final Polish - Apply the Latest Patch Before you begin the sign-off and merge process, ensure you are integrating the best possible version of the software. It is strongly recommended to update your upgrade branch to the latest available minor version or patch for your target release (e.g., if you upgraded to v2.8.2, check if v2.8.3 is now available). These releases contain the latest bug fixes and security updates but no breaking changes, making them low-risk to apply. You can find instructions for this process in the documentation on applying a minor upgrade. Step 2: Gain Stakeholder Sign-Off Before planning the merge, you must get formal approval from all relevant stakeholders. Do not wait until the last minute for this; seek sign-off as soon as individual components are ready for review. Action: Secure approval from the following groups: Product Owners / Business Users: Present your UAT results and regression reports to demonstrate that the functionality is correct and performance is acceptable. Technical Leads / Architects: Get your code reviewed. Your Pull Requests should be approved well in advance of the target merge date to allow time for feedback and iteration. Downstream Consumers: If other teams or applications consume your project's APIs or data outputs, ensure they have been part of the testing and have signed off on the changes. Pro-Tip: If your ETL upgrade was completed first, get it reviewed and signed off while the rest of the upgrade is still being tested. Step 3: Prepare and Educate Your Users Proactive communication is key to a smooth transition for everyone involved with the project. Action: Prepare your user groups for the upcoming change. For Technical Users (Developers): Produce a clear changelog or technical release notes. Inform them of any changes to development environments, deployment strategies, or key data structures that will impact their day-to-day work once the merge is complete. For End-Users (Investigators, Analysts): Provide an updated user guide or release communication that documents any major changes to the User Interface (UI) or application functionality that they will see in the next production release. Step 4: Plan the Merge Logistics This step involves defining the precise technical strategy for merging your code. Action: Define your merge plan, considering project structure. For Split Repositories: Your merge plan must respect inter-repository dependencies. For example, a shared "model" repository must be merged, published, and consumed by the "ETL" and "Apps" repositories before they themselves can be merged. Document the exact order of operations. For Multi-Use Case Platforms: Refer back to your upgrade plan. Ensure that the merge of your upgraded use case will not negatively impact the build or development process for other use cases running on the same platform. Step 5: Coordinate and Execute the Merge to Main This is the final, coordinated event to integrate the upgrade into your main codebase. Action: Follow this checklist for a smooth merge: Set a Merge Date: Align with all development teams on a specific date and time for the merge. Communicate a Merge Freeze: Announce a "merge freeze" for the main branch leading up to the agreed-upon merge time. This prevents last-minute, unrelated changes from introducing conflicts or instability. Prepare a Rollback Strategy: Have a documented plan in place for how you would revert the merge commit (e.g., using git revert) in the unlikely event of a critical, unexpected issue post-merge. Execute the Merge: With all preparations in place, merge your feature/platform-upgrade branch into main. Post-Merge Communication: Announce to all technical teams that the merge is complete. All new feature development must now branch from the updated main branch. With the merge complete, your upgrade is now part of the main line of development. It will be deployed to production as part of your project's next standard release, following your organization's established release playbook.Testing Your Upgrade: Practical Considerations
You have successfully migrated your code to the target version, and you have a stable, building repository. Now, you will execute the Test Plan you created during the planning phase. This guide provides a practical, step-by-step methodology for development testing. The core principle is to start with fast, low-cost tests and progressively move to slower, more expensive, and more realistic validation. Do not jump straight to full data runs. Phase 1: The Foundation - Local and Automated Testing This is the fastest and most important feedback loop. The goal is to catch as many issues as possible on your local machine before ever deploying to a shared environment. Validate Automated Tests: Your first step is to ensure your existing automated test suite is working. Unit & Integration Tests: Run your entire suite of automated tests locally. Fix any failures. These tests are your first line of defense and are crucial for validating that core logic within your ETL and Scoring processes is still sound. Local Runtime Testing: Before deploying, try to run components locally. Example: Starting the Quantexa application services on your local machine can quickly reveal runtime dependency conflicts or configuration errors that were not caught at compile time. Fixing these locally is much faster than in a shared environment. Phase 2: Controlled Environment Testing Once your local tests are passing, it's time to move to a shared development or test environment. The goal here is to perform an initial end-to-end run in a controlled manner. Establish a Stable Baseline: To test for regression, you must have a stable point of comparison. Control Your Code: Ensure the main branch (your baseline) and your upgrade branch are based on the same starting commit. A common practice is to create a temporary release-candidate branch from main to use as a stable, unchanging baseline for comparison. Control Your Data: Use a static, well-understood dataset for this testing phase (e.g., a synthetic or a small, sanitized sample of real data). The input data for your baseline run and your upgrade run must be identical. Produce the Baseline Run: Execute your full end-to-end batch process on the main (or release-candidate) branch using your controlled dataset. Document the results meticulously. This includes: HDFS/S3 output locations. Elasticsearch index names. Batch job runtimes and resource profiles. Produce the Upgrade Run: Execute the same end-to-end batch process on your upgrade branch, using the exact same input data. Compare and Analyze: Compare the outputs from the upgrade run against your baseline. The Statistical Profile Testing Framework (SPTF) is the recommended tool for efficiently comparing batch outputs. Expect some changes. Bug fixes or functional improvements in the new Quantexa version can cause expected deviations. Your job is to validate that all changes are explainable and can be traced back to a specific migration or product change. Any unexplained changes are potential regressions that must be investigated. Phase 3: Full Environment Testing (UAT / Staging) Only after you are confident that most issues have been resolved should you move to a production-like environment with real, full-volume data. Full Regression & SIT: Execute your full test plan in this environment. This includes validating integration with external schedulers, security configurations, and any other real-world integrations. This is your best opportunity to catch elusive, data-specific edge cases. Performance Validation: Run your batch processes against full production data volumes. Compare the runtimes against your pre-upgrade performance benchmarks to ensure there are no significant performance regressions. User Acceptance Testing (UAT): This is a vital part of the release cycle. Block out sufficient time for business users to validate the solution and sign off on the upgrade. Factor in time to investigate and resolve any issues found during UAT and re-test. By following this phased approach, you systematically build confidence in your upgrade, ensuring that by the time you reach the most expensive testing phase, you have already eliminated the vast majority of issues in a more efficient manner.Turn Your Upgrade Plan into Action: Creating JIRA Tickets
This is the final step of the preparation phase. You have a comprehensive Final Upgrade Plan that details every task required for the upgrade. Now, it's time to translate that plan into a structured, actionable backlog of JIRA tickets for the development team. A well-defined backlog is the bridge between planning and execution. It provides clarity for developers on what needs to be done, allows for accurate progress tracking, and forms the basis of your sprint planning. The Process: From Plan to Backlog The core input for this process is the "In Scope Tasks" table from your Final Upgrade Plan. Each row in that table should become one or more tickets in your JIRA backlog. Head Start: To accelerate this process, a set of template JIRA tickets for specific platform version upgrades can often be provided. Ask your Quantexa contact if a template is available for your upgrade path. You can then import and adjust these tickets to fit your project's specific scope. Action: Create Epics for Major Stages Start by creating a master Epic for the entire upgrade (e.g., "Platform Upgrade to v2.8"). Then, if you are performing a multi-hop upgrade, create a sub-Epic for each "hop" (e.g., "Upgrade 2.5 -> 2.6," "Upgrade 2.6 -> 2.7"). This provides a clear hierarchical structure. Action: Create a Ticket for Each Task Go through your "In Scope Tasks" table and create a JIRA ticket (e.g., a Story or a Task) for each line item. A good upgrade ticket is specific, actionable, and contains all the necessary context. What a Good Upgrade Ticket Looks Like Here are examples of how to structure tickets for different types of upgrade tasks. Example 1: Ticket for an Automated Migration Step Title: Run Repository Tool for v2.7 Migration Description: As a developer, I need to run the Repository Tool to apply the automated migrations for the v2.6 -> v2.7 upgrade. Acceptance Criteria: Repository Tool is executed successfully against the feature/platform-upgrade-v2.8 branch. All automated migrations from the v2.7 configuration are applied. The resulting code compiles successfully. Unit tests pass. Any failures or manual interventions required are documented in the Issue Tracker. Example 2: Ticket for a Manual Migration Step Title: [Manual] Upgrade Document Service Response Handling Description: As per the v2.5 migration guide, the response format for the Document Service has changed. We need to manually update our custom UI viewers to handle the new format, which no longer automatically converts Long values to datestrings. Reference: v2.5 Migration Guide#DocService Acceptance Criteria: All custom document viewers are updated to correctly parse and display dates from the new Document Service response. The UI correctly displays document information without errors. Example 3: Ticket for a Tech Debt Task Title: [Tech Debt] Decommission 'Legacy Data Feed' Description: As part of the upgrade, we will remove the unused 'Legacy Data Feed' configuration and all associated code from the ETL process. Reference: Project Backlog Item #1234 Acceptance Criteria: All configuration files related to the legacy data feed are deleted. All code that references the legacy data feed is removed. The main ETL process runs successfully without the legacy code. Planning for Parallel Work While many upgrade tasks are sequential, look for opportunities to parallelize work where possible. This is highly dependent on your project's specific architecture. For example, once the core ETL upgrade is complete and tested, the work to upgrade a separate "Scoring" repository and a separate "Apps" repository can often be done in parallel by different developers. With your backlog created and prioritized, the preparation phase is complete. Your team now has a clear set of tasks to begin the hands-on execution of the upgrade.Get Your Dependencies: What They Are and How to Obtain Them
With your tracking in place, the next step is to gather all the software, tools, code, and access rights you will need to perform the upgrade. Having all dependencies ready and provisioned before starting development is crucial to avoid delays and interruptions. This guide provides a checklist of the key dependencies you will need to acquire. Checklist Item 1: The Quantexa Core Dependency Bundles These bundles contain the core Quantexa product artifacts (libraries and other dependencies) required to build the project against the new versions. Action: Obtain the dependency bundles for every version "hop" in your upgrade path. For example, an upgrade from v2.5 to v2.8 requires the bundles for v2.6, v2.7, and v2.8. How to Obtain: These bundles are not publicly downloadable. You must request them from your Quantexa contact. They will provide you with the necessary files. What to Do: Once received, upload the artifacts to your organization's repository management tool (e.g., Nexus, Artifactory). Ensure that both Maven and NPM artifacts are provisioned. The official documentation provides an example of how to upload to Nexus. Missing Dependencies: If you encounter issues with missing third-party dependencies during your build, it may be because your environment has no access to public repositories like Maven Central. The bundles are designed to be self-contained, so if you believe something is missing from the bundle itself, please raise a post in Quantexa Platform Support on the Community. Checklist Item 2: Component-Specific Bundles (If Applicable) If your solution uses certain Quantexa components like Data Packs, Decision Systems, or QPython, these have their own dependency bundles. Action: Check if your "In Scope Tasks" list from your upgrade plan includes upgrades to these components. If so, you must also request the specific dependency bundles for the target versions of these components from your Quantexa contact. Checklist Item 3: The Repository Tool The Repository Tool is the primary mechanism for automating the code migration between versions in an incremental upgrade. Action: Obtain the Repository Tool executable and the specific migration configuration files for every version "hop". How to Obtain: The tool and its configuration can be provided by your Quantexa contact. Further guidance can be found in the documentation: Download Repo-Tool. Checklist Item 4: Project Example project-example is Quantexa's reference implementation and an invaluable resource. It contains best-practice examples and shows how to correctly apply the migrations for each version. Having a local copy is highly recommended. Action: Get access to the project-example Git repository and, if necessary, transfer a copy to your development environment. How to Obtain: Follow the guidance here to get access: Accessing project-example. Checklist Item 5: Database Migration Scripts (If Applicable) Platform upgrades may introduce database schema changes. For some database technologies (like Oracle), these migrations must be run manually. Action: Discuss with your Quantexa Architect whether manual database migrations are required for your specific technology stack and target version. If they are, you must request the DDL (Data Definition Language) scripts from your Quantexa contact. Checklist Item 6: People and Places (Environments & Access) Software and code are only part of the equation. Ensure the team is fully enabled and the necessary environments are available. Action: Confirm the following before development begins: Team Onboarding: Ensure all developers on the upgrade team have been fully onboarded. System Access: Verify that every team member has the required access to all necessary systems (e.g., Git repository, artifact repository, JIRA, development/test environments). Environment Availability: Confirm that the required development and testing environments have been provisioned and are available for the project. These non-production environments are critical for isolating the upgrade work. With all these dependencies gathered and correctly placed, your team is now equipped with everything it needs to begin the hands-on development work.Implementing Upgrade Tracking: What It Is and Why It Matters
Before you write a single line of code or run the Repository Tool, the first step of the preparation phase is to establish your system of record. Systematic tracking is essential for a successful upgrade, providing a clear, real-time view of progress, issues, effort, and project-specific complexities. This guide provides templates and best practices for setting up the core trackers that will guide you through the execution phase. A well-maintained set of trackers provides three key benefits: Real-time Visibility: Understand exactly where the project stands against the plan, allowing you to identify delays and manage stakeholder expectations. Knowledge Retention: Creates an invaluable audit trail of decisions made, problems solved, and customizations addressed, which is critical for future maintenance and subsequent upgrades. Improved Future Estimates: Capturing the actual effort spent on tasks provides a data-driven basis for more accurate future planning. The Customization & Risk Tracker This is the most critical tracker you will maintain. It documents your project's unique fingerprint, capturing not just code customizations but also environmental or programmatic risks that can impact the upgrade. It should be a living document, reviewed and updated throughout the project. Project Unique Attributes / Areas of Risk Use this table to capture peculiarities of your project that aren't code customizations but still pose a risk or add complexity. Area Description Estimated Maintenance Effort (S/M/L/XL) Estimated Remediation Effort (S/M/L/XL) Risk (Low/Medium/ High/Critical) Comments/Actions Last Reviewed Date CI/CD Uses a custom Jenkins pipeline, not the standard Helm charts. M L Medium The custom deployment scripts may need significant updates for the new version. 2026-03-26 Programme Shares a production data source with Use Case B. S S High Any changes to the shared source must be coordinated and tested with the Use Case B team. 2026-03-26 External Apps The external 'Fraud Dashboard' uses our Search APIs directly. L L Medium API response changes in the new version could break the dashboard. Needs specific integration testing. 2026-03-26 Customizations Use this table to capture any way your solution deviates from standard Quantexa best practice, as per the Customizations documentation. This includes everything from overridden functions to workarounds and significant technical debt. Area (ETL/Apps/etc.) Component Description Relevant Links (JIRA/Confluence) Est. Maintenance Effort (S/M/L/XL) Est. Remediation Effort (S/M/L/XL) Risk (Low/Med/High/Critical) Comments/Actions Last Reviewed Date ETL Post-Processing A custom Spark script runs after the main resolver to enrich entities from an external file. [PROJ-1234] M L Medium This script uses an internal API that may change. It needs to be reviewed against the new version. 2026-03-26 Apps Document Service Custom bean override to change document sorting logic. [PROJ-2468] L XL High This is a high-risk customization. The plan is to remove it and adopt the new out-of-the-box sorting feature. The Issue Tracker This is your central log for all problems, questions, and unexpected behaviors encountered during the upgrade. It prevents issues from being forgotten and helps identify recurring patterns. For detailed guidance, see Issue Tracking Best Practices. Ensure that any potential product bugs identified are fed back to Quantexa Support by raising a new post in Quantexa Platform Support on the Community. The Effort Tracker Tracking the actual effort spent is crucial for refining your project plan and improving future estimates. At the end of each major stage or sprint, update a simple effort tracking table. This provides a clear comparison of planned vs. actual effort. Area / Major Task Development Days Effort Testing & Bug-fixing Days Effort Total Days Elapsed for Acceptance Comments Repository Tool & Automated Migrations 2 3 5 Repository tool failed on two modules, requiring manual intervention. Manual Migration: Document Service 4 2 6 More complex than estimated due to unforeseen impacts on custom User Interface (UI) viewers. Manual Migration: ETL scripts 2 1 3 As expected. By establishing this system of tracking from day one, you create the visibility and data-driven foundation needed to successfully navigate the execution of your upgrade.Preparing For Success
You have successfully completed the planning phase and produced a comprehensive Upgrade Plan. You know what you need to do, how you're going to do it, and how long it will take. The Preparation Stage is the critical bridge between your plan and the hands-on execution of the upgrade. This is where you set up your project for success by establishing the right technical foundations, development processes, and support channels before the first line of code is migrated. A well-executed preparation phase minimizes friction, prevents common pitfalls, and ensures your development team can work efficiently and effectively once the upgrade work begins. Who is this guide for? This guide is primarily for Upgrade Leads and Developers who will be performing the hands-on work. Project Managers will also find the section on turning the plan into actionable tickets essential for tracking progress. What will you accomplish in this stage? By the end of this stage, you will have: Established a robust tracking system for all changes, decisions, and customizations addressed during the upgrade. Assembled all the necessary software, tools, and artifacts required for the upgrade. Established a clear development and environment strategy that allows upgrade work to proceed without halting business-as-usual development. Understood the support paths and processes available to you. Translated your high-level upgrade plan into a backlog of tangible, ready-to-work development tasks. The Preparation Process The following pages provide a step-by-step guide on how to best prepare for the execution of your Quantexa Platform upgrade. Page Description Implementing Upgrade Tracking: What It Is and Why It Matters Learn how to establish a robust system for tracking the upgrade process. This involves logging all issues encountered, the effort spent against tasks, and the details of customizations addressed or introduced. Maintaining these various trackers provides an essential audit trail and helps refine future upgrade estimates. Get Your Dependencies: What They Are and How to Obtain Them The essential next step. This guide covers how to locate and download all the required software versions, Quantexa release artifacts, and tools (like the Repository Tool) that you will need. Upgrading Without Stopping: Development Strategies and Environment Planning Learn how to set up your Git branching strategy (e.g., a long-lived upgrade branch) and plan your environment usage to isolate upgrade development from ongoing business as usual (BAU) work, ensuring both can proceed in parallel. Understanding Support During Upgrades: Where to Turn When Issues Arise Before you encounter a problem, it's vital to know where to go for help. This article outlines the different support channels available, from Documentation and Community to raising a formal Quantexa support ticket. Turn Your Upgrade Plan into Action: Creating JIRA Tickets The final preparation step. This guide shows you how to take the detailed "In Scope Tasks" list from your Upgrade Plan and convert it into a well-defined backlog of JIRA tickets, ready for your team to start working on.Produce Final Upgrade Plan
This is the final step in the planning phase of your Quantexa upgrade. You have now defined your destination, chosen your approach, detailed your scope, estimated the effort, profiled your team, and created a testing strategy. The purpose of this final step is to consolidate all these outputs into a single, formal Upgrade Plan document. This document will be your master guide for the execution phase and the primary communication tool for all project stakeholders. The Anatomy of the Upgrade Plan Document Your final Upgrade Plan document should be structured, clear, and contain all the key decisions and data you have gathered so far. Use the following structure as a template, pulling in the outputs from each of the previous articles in this guide. 1. Executive Summary 1.1. Project Goal: A high-level statement defining the purpose of the upgrade. What is the current version, what is the final target version, and what are the key business or technical drivers for this project? 1.2. Key Outcomes: Briefly list the major deliverables (e.g., "Upgraded Platform to v2.8," "Migration of ETL framework," "Decommissioning of legacy components"). 1.3. High-Level Estimate & Timeline: State the total estimated effort in days and the target completion quarter. 2. Upgrade Strategy & Roadmap This section details the "what" and "how" of the upgrade. 2.1. Target Version: The clearly identified major and minor version for your Quantexa Platform upgrade. This is the output from Identify Your Destination. 2.2. Upgrade Approach: State your chosen approach: Incremental Upgrade or Reset & Rebuild. If incremental, specify the release strategy: Single Go-Live or Staged Go-Live. This is the output from Define Your Upgrade Approach. 2.3. Upgrade Roadmap: Insert the detailed, staged roadmap table you created. This should show the sequence of all major projects (platform upgrades, infrastructure updates, framework migrations) required to reach the target version. This is the primary output from Identify Your Destination. 3. Detailed Project Scope This is the most detailed section of the plan, defining the precise boundaries of the work. It is composed of the tables you built in the Define Upgrade Scope article. 3.1. High-Level Impact Analysis: Insert the "Impacted Components" table (your "heat map"). 3.2. Out of Scope: Insert the table explicitly listing all items that are out of scope. 3.3. In Scope Tasks: Insert the definitive master list table of all migrations, one-off tasks, and tech debt items that are in scope for the project. 4. Effort Estimate & Resource Profile This section outlines the cost and the team required to deliver the plan. 4.1. Effort Estimate: Insert the final estimation table, including the sub-total, contingency, and grand total effort in person-days. This is the output from Estimate The Upgrade. 4.2. Resource Profile: Describe the planned team structure. State that the plan requires an Upgrade Lead and detail the planned team size based on your chosen approach. Note how the plan will leverage a Quantexa Architect/TDO and, if applicable, a Solution Expert. This is the output from Resource The Upgrade. 5. Testing Strategy This section summarizes the plan to ensure a high-quality, stable release, based on the outputs from the Define Testing Requirements step. 5.1. Guiding Principle & Focus Areas: Testing will be risk-based and targeted at the areas of greatest change as defined in the scope (Section 3). The primary focus will be on the components identified as "High" or "Medium" impact, such as ETL and the Mid-Tier. 5.2. Core Testing Activities: The following activities will be performed to validate the success of the upgrade: Testing Pillar Key Activities & Approach Regression Testing Validate batch outputs (ETL, Resolution, Scoring) against a pre-upgrade baseline using the Statistical Profile Testing Framework (SPTF). Manually execute high-value user journey test scripts for the User Interface (UI). Run automated test suites against key API endpoints. Systems Integration Testing (SIT) Perform end-to-end batch runs triggered by the production scheduling system. Verify successful deployment via the CI/CD pipeline, including database migrations. Conduct spot-checks on security and authentication. Non-Functional Testing (NFT) Capture pre-upgrade performance baselines for key batch jobs and API response times. Post-upgrade, execute the same tests to verify that performance has not significantly degraded and remains within acceptable SLOs. 5.3. Multi-Use Case Considerations (if applicable): This plan includes specific tests to validate the interaction between the upgraded Use Case A and the non-upgraded Use Case B, focusing on the shared Centralized Data Source to ensure continued compatibility. Moving to Preparation With this comprehensive plan assembled and approved by your stakeholders, the planning phase is now officially complete. This document is your charter to move into the Preparation Phase. This next phase involves setting up the necessary project structures, environments, and tooling required to begin the hands-on work. Your Upgrade Plan will guide all preparation activities, ensuring the team is perfectly aligned before the upgrade execution begins.Define Testing Requirements
With your scope, estimate, and resource profile defined, the final step in planning is to create a robust testing strategy. The goal of upgrade testing is not to re-test your entire solution from scratch; it is a risk-based exercise designed to build confidence that the upgrade has not introduced any negative impacts and that the platform remains stable. This guide provides a framework specifically for upgrade testing. It should be read in conjunction with Quantexa's High-Level Testing Methodology for a broader understanding of testing best practices. The Guiding Principle: Test Your Scope Your testing effort should be directly proportional to the scope of change you have already defined. There is no need to review release notes or other documentation again; your "Define Upgrade Scope" document is the single source of truth for what needs to be validated. Action: Identify your areas of focus from your scope document. "Impacted Components" Table: This "heat map" tells you where to focus the most testing effort. A component marked with "High" impact requires a more thorough validation than one marked "Low". "In Scope Tasks" Table: Every line item in this table, especially manual migrations and tech debt removal, should have a corresponding test case to validate that the change was successful and didn't have unintended consequences. The Core Testing Pillars Your test plan should be built around three mandatory pillars. The articles below describe how to perform these tests in detail; this section describes what to focus on in an upgrade context. Pillar 1: Regression and Functional Testing Goal: To ensure existing functionality has not been broken by the upgrade. The focus is on validating the outputs and core functions of your solution. For detailed methodologies, refer to the main Deployment Regression Testing and Functional Testing articles. Upgrade Focus: Use the Statistical Profile Testing Framework (SPTF) to efficiently automate the validation of your batch process outputs (ETL, Entity Resolution, Scoring). Pillar 2: Systems Integration Testing (SIT) Goal: To ensure all components of your solution (platform, infrastructure, schedulers, authentication) integrate correctly in a production-like environment. Area Key Activities End-to-End Batch Validate that your external scheduling system can successfully trigger and complete full and incremental batch runs. Deployment & Stability Validate that your CI/CD process can deploy the application services successfully and that any automated database migrations run as expected. Let the services run to ensure stability. Security Spot-check that users can log in via your authentication provider, that API keys function, and that a sample of roles/policies are being correctly enforced. Pillar 3: Non-Functional Testing (NFT) Goal: To ensure the performance, stability, and security of your solution remain within acceptable service level objectives (SLOs). For detailed methodologies, refer to the main Non-Functional Testing Recommendations article. Upgrade Focus: The key is to establish a pre-upgrade baseline for batch runtimes and critical API response times. Your post-upgrade testing should then focus on verifying that there are no significant performance regressions against this baseline. Multi-Use Case Testing Considerations If you are upgrading a platform that hosts multiple use cases, your testing must account for the interaction between them, especially if you are upgrading one use case at a time. Testing Shared Dependencies: If an upgraded use case consumes data from a shared, non-upgraded component (like a centralized data source), you must test to ensure this interaction still works correctly. Testing Upgraded Shared Components: If you are upgrading a shared component itself, it is critical to perform regression testing for all consuming use cases, including those that are not being upgraded in this cycle, to ensure they are not negatively impacted. With a documented test plan covering the core pillars and any multi-use case considerations, you are ready for the final step: Produce Final Upgrade Plan.Resource The Upgrade
With a defined scope and a reliable estimate, the next step is to define the resource profile required to execute the plan. This is not about assigning specific individuals, but about understanding the leadership, skills, and team structure needed for a successful upgrade. The Upgrade Lead The single most important role to plan for is the Upgrade Lead. Whether the upgrade is being performed by a single developer or a larger team, this role is the cornerstone of a successful project. If one person is doing the work, they are the Upgrade Lead. Key Responsibilities: Owns the execution strategy: Breaks down the high-level plan into detailed tasks for the team. Coordinates the work: Manages dependencies and orchestrates the team's activities to ensure they are working efficiently. Drives problem-solving: Acts as the first point of contact for technical issues, responsible for diagnosing problems and leveraging documentation or Quantexa support channels to find solutions. Leveraging Best Practice Expertise While the core team focuses on execution, it's vital to have access to up-to-date Quantexa best practices to ensure the upgraded solution is robust and future-proof. This doesn't need to be a full-time role on the project team. Instead, your plan should include leveraging your organization's existing experts. Plan to engage with your Quantexa-assigned Architect or Technical Delivery Oversight (TDO) at key points in the project. They can provide invaluable guidance on implementing new patterns, validating your approach, and ensuring you are maximizing the value of the new platform version. The Need for Deep Solution Knowledge in a Reset Upgrade For an Incremental Upgrade, a well-documented scope and customization tracker is usually sufficient for the team to work from. However, if you are performing a Reset Upgrade, it is critical that your plan accounts for access to deep expertise on the existing solution. Someone with a strong understanding of the history of the codebase, and the business logic behind its customizations, is essential. This expertise is vital for making informed decisions about which components are valuable and should be rebuilt, and which represent historical technical debt that can be left behind. Planning the Team Size and Structure The temptation on a project with a tight deadline can be to plan for a large team. However, the structure of the work itself is the primary driver of team size. Use the following principles to guide your resource planning: Incremental Upgrades are Often Single-Threaded: An incremental upgrade often involves a sequence of dependent steps that cannot be easily parallelized. For this reason, a plan based on a small, focused team of 1-3 people is typically the most effective. Reset Upgrades Offer More Parallelism: This approach has more potential for parallel workstreams (e.g., one developer on data ingestion, another on scoring). This allows you to plan for a slightly larger team to work effectively, but the work must still be carefully orchestrated by the Upgrade Lead. Validating Your Resource Plan Before finalizing the plan, perform a final check to ensure your resourcing strategy is sound. Action: Validate your intended team profile. Ask yourself the following questions: Does our plan account for a clear Upgrade Lead to coordinate the technical work? Does our plan include checkpoints to engage with our Quantexa Architect/TDO for best practice guidance? If we are doing a Reset Upgrade, have we identified how we will access deep knowledge of the existing solution? Is the planned team size appropriate for our chosen upgrade approach? With a well-defined resource profile, you are ready to move on to the next stage: Define Testing Requirements.
