FAQ: Broken Elasticsearch Head Chrome Extension Workaround
FAQ relevant for: all Academy versions Hi all, Support for Manifest V2 has officially been disabled in Google Chrome, which means the Elasticsearch Head Chrome extension is no longer functioning properly on the VDIs. When you try to add the Elasticsearch Head Chrome extension you may see the following error: "This extension was turned off because it is no longer supported" or this error: "Cannot install extension because it uses an unsupported manifest version. Could not load manifest." We are looking into ways we can resolve this issue, however for now we have a workaround to get the extension working. Error Workaround Here are the steps you can follow to work around this error: Step 1: Navigate to chrome://flags/#extension-manifest-v2-deprecation-warning in the Chrome browser: Step 2: Disable the following settings: Extension Manifest V2 Deprecation Warning Stage Extension Manifest V2 Deprecation Disabled Stage Extension Manifest V2 Deprecation Unsupported Stage Step 3: Restart Chrome. Step 4: Re-add the Elasticsearch Head Chrome extension. This comment details how to add the extension to Chrome. You should now be able to use the Elasticsearch Head Chrome extension as normal! Apologies for the inconvenience, I hope this helps!
FAQ: Elasticsearch "cluster health: Not connected" in Chrome / "Connection refused" script error
FAQ relevant for: all Academy versions Sometimes on the VDIs you will encounter your Elasticsearch being disconnected which will then mean that the data isn't available for easy viewing and it will also lead to errors in your UI. If you try to run e.g. a load Elastic ETL script while Elastic isn't connected then you will get an error, for example: Exception in thread "main" java.net.ConnectException: Connection refused To reconnect the Elasticsearch service you just need to run the following command anywhere in a terminal window on your VDI: sudo systemctl restart elasticsearch.service
Elastic Load Optimization Strategies 📖
Loading data into Elasticsearch can sometimes lead to performance issues, such as slow data loads or loads that fail to complete. The Elastic Load Optimization Strategies guide outlines actionable steps to help improve the performance and reliability of Elasticsearch loads. Key Elastic load optimization strategies: Shard Count Analysis Shards dictate parallelism in Elasticsearch. Adjusting the number of shards for a Document ensures efficient node utilization during loads. Spark Settings Optimize Spark job cores based on Elasticsearch node capacity to enhance indexing performance. Identifying the Problematic Index Pinpoint specific indices causing issues, such as those related to search or a single Entity, for focused troubleshooting. Compounds Table Analysis Analyze the Compounds/DocumentIndexInput.parquet table to uncover further optimization opportunities when issues persist. Compound Partitioning Address large file sizes by repartitioning the compound table during the creation step. Read the full article to explore these strategies and ensure faster, more reliable Elasticsearch loads (login required): Elastic Load Optimization Strategies - Quantexa Community Projects may encounter challenges with performance when loading data to Elasticsearch. This may present in the form of excessively slow loads or loads that fail to complete. The following outlines a series of steps projects should consider when trying to improve performance and reliability in such cases: Shard Count…
Common Elastic Loader errors
For common elastic loader errors, such as load elastic job failing, read Common Elastic Loader errors (login required) on our Docs site. You'll find some common Elastic Loader failure cases, and how to address them. The errors apply to both the Resolver-Search Elastic Loader and the Generic Elastic Loader. These include: Connection issues Performance issues Elasticsearch is overloaded Spark is running out of memory Count validation failure Data could not be indexed Did you know you can also click the elastic-loader tag and then filter to find Questions with an answer that is Accepted by the Community?Why Does Entity Quality Matter? & Best of the Community from March
March Top Picks Why Entity Quality Matters 🔐login required Enter our latest competition: The Knowledge Exchange 📝 New Education Programs: Scala & Spark Bootcamp and Quantexa Data Engineer Velocity Program Quantexa & Xander Talent - New Education Partnership 🤝 Elevating Data Management: Unveiling the Pillars of a Trusted Data Foundation with Quantexa Latest from the Community Library 📖 A day in the life of a... Senior Learning Designer A day in the life of... an Academy Trainee How To Test Your Upgrades 🔐login required Updates to Quantexa Supported Versions 🔐login required Upcoming events 🗓️ 3rd May Community Connect 👋 Join for a demo of top Community features. Best of Q&A ✅ Unable to load Batch Scores to Elastic 🔐login required How to fix image style for investigation icon in the Qx UI? 🔐login required Error creating bean with name 'springSecurityFilterChain' 🔐login required New & Popular Ideas💡 Usability increase using 2 screens for investigations 🔐login required Changing the default settings for Graphic Filters in Data Viewer 🔐login required Make updates to metadata.parquet optional 🔐login required In case you missed it 📣 Welcome to Quantexa 2.6 | 2.6.0 Release Announcement 🏆 Badge of the Month: The Name Dropper Badge Community quick links 🚀 Submit and vote for Ideas in our Ideas Portal 🗣️ Join one of our Specialist User Groups: FinCrime, Insurance, Data Management & KYC 📚️ Browse blogs, articles and guides in our Community Library
Welcome to Quantexa 2.3 | 2.3.0 Release Announcement
In Quantexa 2.3, you will find: Entity Store, a new component in our set of Entity Resolution capabilities; Introduction of Security Model V2, a new way to manage Role-Based Access and Control within The Quantexa Platform; Enhancements to Assess including Path Ranking, a new way to define paths; Support for Elasticsearch 8 and Spark 3.2. Entity Store In 2.3.0, we are introducing Entity Store in beta. Before 2.3.0, all interactions with the Quantexa UI and the Quantexa Mid-tier APIs depended upon resolving Entities on-the-fly (dynamically). The Entity Store allows a persisted or materialized view of Entities to be stored in the system. In this first release, the core Entity Store will load a full set of pre-resolved Entities produced by Batch Resolver, into Elasticsearch. The Entity Store can then be used by Explorer to enable querying of the underlying entities, as a Cache for Resolver to improve performance, and to support the new Entity REST API. Security Model V2 We have introduced a new framework for authentication and authorization across the platform. This gives the platform greater control over the data and features that a user can access and enables easier integration with Identity Providers (IdPs). End-users no longer need to interact directly with low-level Quantexa Roles. They are now able to share their work with Groups that are logical to their organizational structure. For example: UK Fraud Investigators. Using a new User Management Screen, you can now provision users and Groups, collections of users that can be assigned Roles and Dynamic Privileges, to The Quantexa Platform from Identity Providers simply and easily. Path Ranking It is now possible to specify the rules for defining the relative importance of paths, a chain of Documents and Entities that connect two Nodes, when writing Network Scores with Path Ranking, which make the Network easier to analyze by signaling the most significant areas to explore further. Dual Context Sources Dual Context Sources enable scoring logic to be designed that will work in both batch and dynamic, and provides a config-based tool that generates Source steps for both pipelines. This simplifies the deployment of a build with a dual architecture. Support for Elasticsearch and Spark Quantexa now supports Elasticsearch 8 across the platform, except for Offline Indexers and Spark 3.2 is now supported across the platform. Warning: Spark 3.0 is deprecated and support will be removed completely in the next release of the platform. Spring Boot upgrade Spring Boot has been upgraded to version 2.6.8. This results in faster startup times and solves a number of security vulnerabilities found in older versions of Spring Boot. Documentation Site Glossary We have completely revamped and reworked our Documentation site glossary, adding almost 100 new terms. Explore the Glossary to find out more. Other highlights For more control and flexibility when running ETL, Quantexa has has added the ability to generate Resolver Search Loader and Compound Creator scripts at the point where users define a Root Model in Data Fusion. List inputs have been introduced for Entity Attribute functions, to allow for more flexibility and customizability, granting users more ways to define Entity Attributes in Data Fusion, and cater to a wider range of data models. You are now able to collapse the Query Builder in Explorer. This is the next step in improving the user experience of Explorer, allowing users to focus on the results of the query. You will find the full set of Release Notes on the Quantexa Documentation site. If you are unable to access them, you will need to get a user with access to submit a Documentation site access request through the Quantexa Support Portal.
Welcome to Parsers 4.1 | Release Announcement
We are excited to announce the release of version 4.1 of Quantexa's Standard Parsers. This release focuses on improving the integration with Fusion UI (look out for exciting 2.6 release announcements coming soon) and improvements of the file structure of configuration files. This release includes the following highlights, which are detailed below. Consistency of configuration files - general improvements to Parser and lexicon configuration and files have been introduced to make sure the way you use these files is consistent across all available Parsers. This will make your configuration easier to understand and simplify the process of making future modifications. To minimise redundant data storage in Elastic Search, you can now exclude business standardisation terms that aren’t used in areas such as exclusions for Entity Resolution. Similarly, you can now choose to parse multiple names or just a single name for the Individual Parser to reduce your Elastic Search footprint. For the Telephone Parser you can now specify conditional parsing rules to increase the output accuracy. For example, if you have more specific parsing rules for UK telephone numbers, you can now use country code to parse these telephone numbers differently to the default telephone parsing behaviour. Note: There are no changes in this release that affect the output of parsing.October Community Digest 🎃
Welcome to this spooktacular edition of our Community Digest, where we gather the most thrilling posts from our members just in time for Halloween 👻 Public content: 💬 Tips & Tricks for Managing Large and Complex Networks - Discussion 📅 Join our Monthly Community Connect on November 10th - Event 📣 Quantexa's Chris Bagnall wins ACAMS Today Article of the Year 2023 - Announcement 📚 Understanding Human Trafficking - Blog 💬 Elasticsearch and Why We Use It - Discussion 📚 Automatic Data Cleaning Through Data Normalisation and Statistics - Blog 📅 Unlocking the Power of pKYC: Smarter KYC Processes EMEA and APAC webinars 🏆Community Competition - Refer 5 Colleagues to Win! 📚 Elasticsearch Considerations for Quantexa - Blog 💬 Perspective: Addressing SEC-Identified AML Program Deficiencies at Broker-Dealers - KYC Group Discussion 📚 Using Data Fusion for the first time - Blog Members content (log in required): 💡Make dropdowns in Explorer charts searchable and sort options alphabetically in multiple languages - Idea ✔️Load Elastic not connecting - Academy Q&A 💡Auditing search viewer click event for audit monitoring - Idea ✔️When aggregating entity scores, only one appears in the customer scorecard - Q&A 💡Running out of screen real estate in the Investigation view - Idea Community quick links: 💡 Submit and vote for Ideas in our Ideas Portal 🗣️ Join one of our Specialist User Groups: FinCrime, Insurance, Data Management & KYC 📚️ Browse blogs, articles and guides in our Community Library New to the Community? Sign up for a Community Tour 😊Elastic Collection
Below you can find links to all the articles and discussions relating to Elastic in the Community. Elasticsearch, or Elastic, is a near real-time, distributed storage, search, and analytics engine. Since the beginning, Quantexa has used Elasticsearch to store and query the data we ingest into the Quantexa platform, as we knew Search was going to be such a central feature of the platform. 🗒️Articles available to everyone Elasticsearch Considerations For Quantexa Elasticsearch and Why We Use It 🔒Log in required Useful Elasticsearch API Calls Bookmark this thread and be notified whenever we publish a new article on Elastic. To bookmark a thread, click the chevron icon next to the title of the post.
Useful Elasticsearch API Calls
, Solutions Architect, outlines how to manage Elasticsearch (Elastic) clusters and indices which are crucial for maintaining a Quantexa implementation. This article provides an overview of common Elasticsearch API calls that can be used for cluster and index management. Key topics: Cluster Management: Retrieve cluster statistics using the _cluster/stats endpoint. Index Manipulation: Create an index with specific settings. Delete an index. Open and close an index. Enable read/write on an index. Reindex data from one index to another. Force a merge operation for better read performance. Move shards within the cluster. Alter the number of replicas for redundancy. Disable and re-enable shard allocation. Index Interrogation: List aliases. List indices, including their health and status. List shards and their details. List segments within an index. Retrieve index mapping. List unassigned shards. Check the progress of a force merge operation. Index Entry Manipulation: Add an entry to an index. Nodes: List nodes in the cluster, including their roles and resource usage. List nodes with detailed queries. Search: Perform a search query within an index using cURL requests. Tasks: List active tasks within the cluster. List detailed task information. This serves as a practical guide for managing Elasticsearch clusters and indices in the context of a Quantexa implementation, providing essential commands to effectively maintain and optimize the Elastic search environment. Useful Elasticsearch API Calls - Quantexa Community Managing your Elasticsearch clusters and indices is an important part of building and maintaining your Quantexa implementation. In this blog we have outlined and explained some of the more common Elasticsearch API calls that you may need. The example cURL requests below assume you are on an Elastic node. If that’s not the…
