This blog is about getting a simple Dynamic Scoring pipeline off the ground in Quantexa version 2.0.0, to get some Scores showing in your User Interface (UI).
So what’s Assess Dynamic Scoring all about?
Figure 1. Assess Dynamic Scoring
Compared to getting basic Dynamic Scoring working in Scoring Framework 1 (SF1), it’s more involved doing the same using Assess, but you have more flexibility as you build out a more complex Scoring model, and there is far better documentation and support.
The main difference is that you now must configure an end-to-end Dynamic Scoring Directed Acyclic Graph, commonly known as DAG, from Elasticsearch (ES) source to a final step, adding your Scores to the UI. Using Assess, you’ll think of Dynamic Scoring as a pipeline, as opposed to a list of Scores being applied in the UI as per SF1.
Notable improvements are the new Score description rendering, the ability to have multiple Scorecards of different levels present in the UI at once, and the DAG visualizations
— Tom Triviais, Senior Data Engineer at Quantexa
Why is the feature useful?
In general, Dynamic Scoring allows you to execute Scores on-demand. This means that you can submit Documents, Entities, and Networks to Dynamic Scoring as they are built. Assess still delivers the same quality of scores you'd expect from SF1, but goes about it in a slightly different way.
Compared to a Batch approach, which can take minutes, hours or even days to process data, Assess Dynamic Scoring updates scores whenever the Network is updated, allowing you to automatically see Scores, straight away, in the UI.
In the context of Scoring with Assess, it's particularly useful since factors can cause specific scores to change, and a dynamic process will make sure the information you are referring to stays as up-to-date as possible.
A pipeline approach, which is one of the main differences from SF1, is a more stable, efficient and reliable approach to the Scoring process.
What did you find out during implementation?
Figure 2. A simple Dynamic DAG, with dummy Scores and a Network-level Scorecard
We started out trying to set up a simple Network-level Scorecard, combining Entity and Document Scores, which requires a few straightforward steps. Experience with Assess Batch Scoring helps, as there are shared concepts, but isn’t essential – it just takes some time to get your head around.
Some other things which you can also do, which are obvious improvements, include the new Score description rendering, the ability to have multiple Scorecards of different levels present in the UI at once, and the DAG visualizations (which is excellent reference material for understanding the flow of a Scoring pipeline).
If you’re starting from scratch, you consider first working on a 'shake out', getting a simple version of your intended Scoring pipeline functioning and displaying in the UI, then iterating, adding in Scores and subsections of the pipeline (for example, in the DAG, starting with just the Customer Document pipeline, then once displaying in the UI as intended, adding in the individual Entity part of the DAG).
Design considerations
Estimation
Given the differences to SF1 Dynamic Scoring, which most engineers will be familiar with from the Academy, consider allocating extra time as the team will need to learn how to use Assess – no aspects are too difficult, just different. See the following section for Implementation tips. There’s more code required to add and be reviewed than in SF1 (although nothing overly complex) so factor this in too. If you can, it would be beneficial to line up a contact who is familiar with Assess before commencing development, in case the team needs some support.
What to score?
For each Document or Entity type you want to score, you need to implement a 'pipeline' (subsection of DAG) to process and apply Scores of that type. It may be worth breaking these down into separate tickets per Entity/Document type and building the DAG up iteratively.
Scorecard(s)
You don’t need to consider a Scorecard to get Dynamic Scores showing with SF1 – with Assess you do, and can implement multiple different Scorecards to be applied at once in the UI. This is a great new addition and requires more design consideration than SF1.
Related node highlighting
This visually highlights other nodes which might be worth investigating, and is straightforward to implement with Assess, consider including such requirements in your tickets for Scores.
End-to-end testing
An end-to-end test should be implemented and any additions to the DAG should be included and tested as a part of this, as well as standard unit tests for Scores as per SF1.
Implementation tips
Review the Scoring DAG
When using any example Dynamic Scoring code for reference, it’s very useful to look at the corresponding DAG to visualize the pipeline and therefore better understand the Scoring steps – particularly for more complex pipelines like those in Project Example. Likewise for your own project, the DAG is a key tool – see the Scoring DAG documentation for more details.
Perform a Dynamic Scoring test
Implement an end-to-end Dynamic Scoring test from the get-go (think of it as a Dynamic Scoring integration test). This can compile and run your DAG; you can test each step and it’s much faster than building app-scoring and running your apps only to find issues with the DAG. It might be challenging for a junior engineer to get set up, but worth the investment in the long run. There are examples in Project Example and Scoring Framework getting started. Inputting the relevant data to trigger your Scores can be a convoluted process, so watch out for issues in that aspect.
Run a clean build
You might encounter intermittent build issues when building app-scoring or scoring-steps module such as Execution failed for task :app-scoring:bootJar. Main class name has not been configured and it could not be resolved (possibly related to the plugin). These can typically be resolved by running a clean build, for example: ./gradlew clean app-scoring:build.
Examples
There are examples in Project Example and Scoring Framework getting started sections of the Documentation site.
Build information
Version 2.0.0 TQP, May 2022
Additional Resources
Did you know that you can log in (or sign up) to the Community to unlock further resources in the Community and on our Documentation site?
