How to Perform an Incremental Upgrade

You have completed your planning and preparation, and you are now ready to begin the hands-on execution of the upgrade. This guide provides the step-by-step technical process for performing a single "hop" of an incremental upgrade (e.g., migrating from v2.5 to v2.6).

The fundamental principle of an incremental upgrade is to migrate your project one major version at a time, ensuring the codebase is stable and validated at each stage before proceeding to the next.

The Key Tool: The Repository Tool

The Quantexa Repository Tool will be the primary engine for your upgrade. It uses a file generator called PLOPjs and code-modification tools like Scalafix to automate a significant portion of the migration effort. While powerful, this tool does not cover every scenario, and manual changes may therefore be required.

The Workflow for a Single Upgrade Hop

For each hop in your upgrade roadmap (e.g., from v2.5 to v2.6), you will follow this four-phase process.

Phase 1: Automated Migrations

The first step is always to let the automated tooling do the heavy lifting. This is a highly scripted process. For detailed instructions on the commands, refer to the documentation on Running the repository tool.

1. Configure the Tool: Locate the migration-config.json for the version hop you are performing. Update the projectPath to point to your repository.
2. Run Scalafix Migrations: Execute the Scalafix migrations first.
3. Run Plop Migrations: Execute the relevant Plop migrations. It is best practice to run these one by one to create a granular commit history.

Pro-Tip: If your project uses split repositories (e.g., separate ETL and Apps repos), only run the migrations relevant to the repository you are currently working on.

Phase 2: Manual Migrations

With the automated changes applied, you will now execute the manual tasks from the backlog you created in the preparation phase. Your JIRA board is your guide for this phase.

1. Execute Manual Migration Tickets: Work through the JIRA tickets you created for the mandatory manual migrations. Each ticket should contain the context and a link to the specific documentation needed for that single task.
   
   
2. Address Customization Tickets: Work through the tickets related to your project's customizations. These tasks involve reviewing how the upgrade has impacted your custom code and applying the necessary fixes to make it compatible.
   
   
3. Execute Optional Migration Tickets: If you decided to include any optional migrations in your scope, execute those tickets now.

Pro-Tip: Make small, specific commits for each distinct manual change (e.g., "Fix custom scoring function for v2.6 API change"). This creates a clean, traceable history. Refer to project-example to see how the same migrations were applied in Quantexa's reference project.

Phase 3: Compile and Validate

With the automated and manual changes applied, this phase is focused on compiling the code and validating its integrity by running the automated test suite.

1. Compile the Code: The first action is to run a full build of the repository. If any compilation errors arise from the applied migrations, resolve them. These are typically caused by API changes or updated method signatures that impact custom code.
   
   
2. Validate with Unit Tests: Once the repository compiles successfully, run your full suite of automated unit tests. Address any tests that are failing as a result of the upgrade changes.

A successful, clean build with all unit tests passing marks the end of the development work for this hop.

Phase 4: Intermediate Validation (Optional but Recommended)

Before moving to the next hop, it is highly recommended to perform some level of intermediate validation to catch issues early.

• Run your automated integration tests.
• Perform a small, local ETL run to ensure the core process works.
• If practical, deploy the applications locally to check for runtime errors.

Repeat and Finalize

Once you have completed this four-phase process for one hop, you repeat the entire workflow for the next hop in your upgrade roadmap (e.g., v2.6 -> v2.7).

After the final hop is complete and you have a stable, building repository on your target version, the hop-by-hop development phase is over. You are now ready to proceed with the full Development Testing as outlined in your test plan.