New guide: Using the Entity Quality Underlinking (EQU) tool for the first time 📖
The Entity Quality Underlinking (EQU) tool is a powerful resource for tuning and monitoring Entity Resolution. Using the Entity Quality Underlinking (EQU) tool for the first time is a detailed guide to implementing the Entity Quality Underlinking tool, including its design, implementation tips, and practical use cases. Why is the Entity Quality Underlinking Tool useful? The Entity Quality Underlinking Tool helps you: Identify underlinked Entities and analyze root causes through manual examination in the UI. Measure the extent of underlinking over time, especially when tracking this metric in Production. Adjust Entity Resolution templates to address Overlinking issues identified earlier. What does the Entity Quality Underlinking Tool do? Monitoring and Tuning: The Entity Quality Underlinking Tool supports both tuning iterations and ongoing Entity Resolution monitoring. Analysis: It observes the similarity of Entity Elements and identifies potentially underlinked Entities. Output: The Entity Quality Underlinking Tool generates: Summary Statistics for tracking improvement or ongoing performance metrics. Potentially Underlinked Entities for investigation in the User Interface (UI). What’s in the guide? Step-by-step instructions for implementation. Design considerations for effective use. Tips to ensure smooth implementation and accurate results. Read the full article for a comprehensive understanding of how to integrate the Entity Quality Underlinking Tool into your Entity Resolution processes (login required): Using the Entity Quality Underlinking (EQU) tool for the first time - Quantexa Community This article details the implementation of the Entity Quality Underlinking (EQU) tool, developed to assist when tuning Entity Resolution. What is EQU? The EQU tool is used as part of Entity Resolution Tuning and BAU Entity Resolution monitoring. It observes the similarity of your Entities' Elements and identifies whether…
Elastic Load Optimization Strategies 📖
Loading data into Elasticsearch can sometimes lead to performance issues, such as slow data loads or loads that fail to complete. The Elastic Load Optimization Strategies guide outlines actionable steps to help improve the performance and reliability of Elasticsearch loads. Key Elastic load optimization strategies: Shard Count Analysis Shards dictate parallelism in Elasticsearch. Adjusting the number of shards for a Document ensures efficient node utilization during loads. Spark Settings Optimize Spark job cores based on Elasticsearch node capacity to enhance indexing performance. Identifying the Problematic Index Pinpoint specific indices causing issues, such as those related to search or a single Entity, for focused troubleshooting. Compounds Table Analysis Analyze the Compounds/DocumentIndexInput.parquet table to uncover further optimization opportunities when issues persist. Compound Partitioning Address large file sizes by repartitioning the compound table during the creation step. Read the full article to explore these strategies and ensure faster, more reliable Elasticsearch loads (login required): Elastic Load Optimization Strategies - Quantexa Community Projects may encounter challenges with performance when loading data to Elasticsearch. This may present in the form of excessively slow loads or loads that fail to complete. The following outlines a series of steps projects should consider when trying to improve performance and reliability in such cases: Shard Count…
An Introduction to Performance Testing a Quantexa Deployment with Gatling 📖
This article introduces using Gatling, a Scala-based open-source performance testing tool, to evaluate a Quantexa mid-tier deployment. It explains how to write and run test scenarios simulating user behavior via Quantexa’s REST APIs to assess system performance. Key considerations include: Ensuring consistent testing environments Understanding Gatling's limitations in simulating real user behavior The importance of observability tools. The article also discusses how proper scenario design and interpretation of results are crucial for meaningful insights into deployment performance. Read the full article here (login required): 2. An Introduction to Performance Testing a Quantexa Deployment with Gatling - Quantexa Community This article provides a high-level introduction to Gatling testing a Quantexa mid-tier deployment. What is Gatling? Gatling is a Scala based, open-source performance testing framework. For more information, see the introduction documentation or this blog post. Why use Gatling with Quantexa? You can use Gatling to…
Tips & Tricks for Managing Large and Complex Networks - Update for 2.5 & 2.6
As an investigator you may have found that the use of entities and networks are helpful for identifying potential risk in the investigation process, using connections across documents and entities. However, seeing all the available data connected by way of transaction flows, trades, direct links or indirect links can present its own challenges. Substantial amounts of data in a network can sometimes be difficult to navigate. Where do you start? How do you find how many hops to expand then stop? What do you do with the information which is no longer useful to the investigation? Read the full article Tips & Tricks for Managing Large and Complex Networks for the latest functionality which can help simplify how to view and use networks to their full potential. Tips & Tricks for Managing Large and Complex Networks - Quantexa Community As an Investigator, you may have found that entities and Networks are helpful for identifying potential risk in the investigation process, using connections across documents and entities. However, seeing all the available data connected through transaction flows, trades, direct links, or indirect links can present its own…
Scoring Concepts: Alerting and Re-alerting 📖
Alerting is the name for the process that comes after Scorecard creation and before Task loading into a Quantexa Deployment. Alerting decides which Subjects in the Scorecard output should alert to the end-users. Re-Alerting is an Alerting process that occurs after the first Alerting cycle, when new Scorecard data becomes available. Re-alerting logic compares a Subject's Scorecard output with all previous Scorecard outputs. The aim of this process is to ensure that there is new material risk that an Investigator would like to review. It is strongly encouraged that all deployments with Batch Scoring implement Alerting. Read about the Alerting Framework, Alerting Threshold, and Score Types in Scoring Concepts: Alerting and Re-alerting. Read the full article (login required): 4. Scoring Concepts: Alerting and Re-alerting - Quantexa Community This article builds upon the concepts introduced in Scoring Concepts: Scoring Levels and Scorecards. Alerting is the name for the process that comes after Scorecard creation and before Task loading into a Quantexa Deployment. Alerting decides which Subjects in the Scorecard output should alert to the end-users. Re-Alerting…
Common Elastic Loader errors
For common elastic loader errors, such as load elastic job failing, read Common Elastic Loader errors (login required) on our Docs site. You'll find some common Elastic Loader failure cases, and how to address them. The errors apply to both the Resolver-Search Elastic Loader and the Generic Elastic Loader. These include: Connection issues Performance issues Elasticsearch is overloaded Spark is running out of memory Count validation failure Data could not be indexed Did you know you can also click the elastic-loader tag and then filter to find Questions with an answer that is Accepted by the Community?
Tips and Tricks for Understanding Entity Lab
Entity Lab can be quite complex to understand and review when searching for Entity Resolution problems. To help with this, we've compiled some tips and tricks for what to watch out for that may help you quickly understand the health of your entities as well as some common shapes you may encounter, including: Signs of Possible Overlinking Signs of Underlinking Possible Aggregation Issues Read the full article: Tips & Tricks for Understanding Entity Lab (login required) Tips and Tricks for Understanding Entity Lab - Quantexa Community Entity Lab can be quite complex to understand and review when searching for Entity Resolution problems. Below are some tips and tricks for what to watch out for that may help you quickly understand the health of your entities as well as some common shapes you may encounter. Signs of Possible Overlinking Look for lots of…
💭 All things Glyph!
Hi Community, We wanted to help direct you to some useful Q&A relating to Quantexa Glyphs: Glyph sizes Glyph with total score condition I'm not getting the glyph shown for the document. How to get glyph icon working correctly Watchlist Contextual Search glyph error in v2.0.3 On a Contextual Search node, is possible to apply custom glyph logic based on the search result? Cannot find name 'SigmaHighlightColor' Final assessment : Glyphs Re: Adding glyphs : Final Assessment Custom Glyphs not rendering in UI after upgrade to v2.0 Glyphs not displaying text correctly in UI Re: [Module 6.1] UI customizations — what's expected? (accessible to customers & partners only - log in required)📚 Kafka Data Ingest
Kafka Data Ingest handles near real-time ingestion of raw Documents into the platform. This enables you to perform ad-hoc Document ingestion and implement larger-scale integrations with event-streamed data sources. Kafka Data Ingest consists of two sets of services, each running within a separate application: Record Extraction service: The Record Extraction service handles the Cleansing, Parsing, and extraction of Records from raw Documents. Document Ingest service: The Document Ingest service loads extracted Records to Elasticsearch for use within Quantexa services and the UI. Read more about Kafka Data Ingest on the Documentation site. Note: Kafka Ingest was referred to as Kafka Loader before version 2.1.🔎 Resolver Config
Resolver config (or resolver-config) is the technical backbone of entity resolution. 🔎 Interested in learning more? Why not explore these Community posts (did you know if you're logged in, you'll unlock even more resources): Read about resolver config in the best practice library section on Entity Resolution Explore Community posts related to resolver config: Resolver config All posts tagged with resolver config Further documentation can be found on the Docs site: Resolver configuration reference Resolver YAML configuration Resolver JSON configuration Resolver Elasticsearch configuration Resolver security Resolver custom Attributes Resolver Attributes migration
