FAQ: Elasticsearch "cluster health: Not connected" in Chrome / "Connection refused" script error
FAQ relevant for: all Academy versions Sometimes on the VDIs you will encounter your Elasticsearch being disconnected which will then mean that the data isn't available for easy viewing and it will also lead to errors in your UI. If you try to run e.g. a load Elastic ETL script while Elastic isn't connected then you will get an error, for example: Exception in thread "main" java.net.ConnectException: Connection refused To reconnect the Elasticsearch service you just need to run the following command anywhere in a terminal window on your VDI: sudo systemctl restart elasticsearch.serviceFAQ: I can't find Addresses or Individuals when I search in my UI!
FAQ relevant for: all Academy versions If you perform a search in your UI and it only returns eg. Business entities but not Individuals or Addresses, then there are a few things you can check to resolve this issue. Firstly, remember that when you perform a general search (red box below), it will search within any field sets (orange box) you have configured in the search definitions of the resolver config. If you have configured these search definitions for all entity types, then also double check for any spelling errors in the resolver config in terms of fields that you are pointing at in the underlying data - these searches go off of fields in the cleansed Document Data Model. The other place to check is that your elastic has the data you expect in it - do you have address and individual data there? If not, then you may want to review your ETL pipeline, and check out this related FAQ. Lastly - you may need to refresh the security permissions to be able to access these additional Entities. You can do by running: ./drop_recreate_databases.sh In your home directory on the VDI (while the UI isn't running). After this, you should see the additional search fields and be able to find these entities. Let us know in a new post if you are still having issues after performing these checks!FAQ: I'm trying to run a script and seeing an error about "Unrecognized option: -s"
FAQ relevant for: all Academy versions If you are trying to run a Scala script (e.g. ImportRawToParqet) and you see the above mentioned error, the main cause of this is that the program can't find the relevant JAR file to run. You may also get an error saying something like " .jar not found" or "jar does not exist, skipping." The solution to this error is to build the relevant JAR file(s) as specified in the Spark Shell script that you tried to run the Scala file with. For example, in the "runQSS.sh" script of the academy task project, you will see that this requires two JARs: the data-source-all Project and Dependency shadow JARs. By building these two JARs and verifying that they are in the correct location and have the correct name to match the full file paths in the relevant Spark Shell script, this should fix the issue. Let us know in a new post if the above solution didn't fix the issue for you!FAQ: I'm missing data in Elasticsearch / my number of docs are wrong
FAQ relevant for: all Academy versions If you have completed the ETL pipeline stages of your project and uploaded the data to ElasticSearch, then when checking your indices in the ElasticSearch Head plugin on Chrome you should have numbers similar to the picture below (to get a bigger version of the image, right click it and chose the option to open it in a new tab). Note: If your numbers vary a little bit from these, for example having 152k address instead of 148k, then that's ok - the numbers will change a little for the resolver indices (Individual/Address/Business) based on the compound keys you have imported in the respective *.qentity Fusion config files. If your numbers are significantly different to this, then you will want to go back through your ETL pipeline and carefully check each stage to see if there is somewhere that you lose the data along the way. A good way to approach this problem is to work forwards from CreateCaseClass and check the output of each stage to find the problem area. You should also use the counts in ElasticSearch to guide you - for example if you have only half the number of businesses listed above, and no individuals, it lets you know that you probably haven't joined your Third Parties onto the ICIJ document properly, and so you would want to go back and double check how you have done this join and on what fields. Specific points to consider: Have I correctly parsed all of the necessary fields in my qmodel files? Have I used the correct type of joins in CreateCaseClass, and have I joined on the correct fields? Have I outputted the correct Dataset at the end of CreateCaseClass? Have I loaded up the DocumentDataModel.parquet (the output of CreateCaseClass) into a Spark-Shell to check the output there? Have I correctly identified and defined all relevant start paths in my qentity files? Do I have a good range of compound keys for each Entity? If you are convinced that you have done all of the above correctly then you can try to clear the data from ElasticSearch, restart the service and then re-upload the data to Elastic using the following three commands: curl -X DELETE 'http://localhost:9200/_all' sudo systemctl restart elasticsearch.service ./runQSS.sh -s com.quantexa.academy.task.icij.model.etl.IcijLoadElasticScript -c ../external.conf -r elastic.icij📢 Welcome! Please Read Before Posting
Hey there, and welcome to the Academy Discussion space of the Quantexa Community! 😀 The purpose of this space is for people to be able to get help and information as they journey through the Quantexa Academy! Encountered a bug which you can't see how to fix? Have a query about a specific area of the Academy task? Need help with understanding some of the key concepts? Let us know by asking a question! Before posting a question however, please review the following article and video around debugging and getting the most out of the Community, which you can find here: https://community.quantexa.com/kb/articles/264-1-how-to-maximize-value-from-community-support Remember, vague discussions are hard to act on so please provide as much detail as you can! If you have any questions about access issues (e.g. you don't have access to LearnUpon or the Quantexa Docs site) then please email training@quantexa.com rather than posting here, thanks! 😀
FAQ: Broken Elasticsearch Head Chrome Extension Workaround
FAQ relevant for: all Academy versions Hi all, Support for Manifest V2 has officially been disabled in Google Chrome, which means the Elasticsearch Head Chrome extension is no longer functioning properly on the VDIs. When you try to add the Elasticsearch Head Chrome extension you may see the following error: "This extension was turned off because it is no longer supported" or this error: "Cannot install extension because it uses an unsupported manifest version. Could not load manifest." We are looking into ways we can resolve this issue, however for now we have a workaround to get the extension working. Error Workaround Here are the steps you can follow to work around this error: Step 1: Navigate to chrome://flags/#extension-manifest-v2-deprecation-warning in the Chrome browser: Step 2: Disable the following settings: Extension Manifest V2 Deprecation Warning Stage Extension Manifest V2 Deprecation Disabled Stage Extension Manifest V2 Deprecation Unsupported Stage Step 3: Restart Chrome. Step 4: Re-add the Elasticsearch Head Chrome extension. This comment details how to add the extension to Chrome. You should now be able to use the Elasticsearch Head Chrome extension as normal! Apologies for the inconvenience, I hope this helps!
Discover LearnUpon's (Our LMS) Fresh New Look
Hello Learners, Exciting changes are coming to our Learning Platform! Quantexa Academy is pleased to announce that our Learning Management System is getting a new look to enhance your learning experience. This upgrade will offer a more intuitive design, improved navigation, and a more engaging interface. Here are the changes you can expect: New User Dashboard - Home and My Learning page: A simplified, easy-to-navigate homepage that helps you access learning content quickly. Content Details Page - Course/Learning Path: A more engaging and visually rich way to explore course/learning path details and progress. Get ready for an engaging, seamless, and user-friendly learning experience with the new User interface. FAQs 📅 When will the new UI be released? The new user interface (UI) will be activated on 2 June 2025. 🙋♂️🙋♀️Will this update affect my program history? No, this update is purely visual, and your history of completed courses will not be affected. How to use the new UI: To help you navigate the new UI, please refer to this guide to familiarize yourself with the changes. Note: All current images used are stock examples provided by LearnUpon and will be replaced once the updates are implemented in our Learning Portal. 1. New Dashboard Home Page The Home Page is a new, redesigned dashboard allowing you to view all your learning content at a glance. You can easily find the most essential materials, upcoming trainings/live sessions and track your progress. My Learning Page My Learning page displays all your course/learning paths: past, present and future. Each course is represented by a card that summarizes essential information in a compact format, such as the course title, course image, course or learning path label, etc. The Home and My Learning pages will replace the current learner dashboard. 2. Content Details Page: Course/Learning Path The new content details page is designed to create engaging and enriching experiences for learners. It features a refreshed visual layout that enhances how learners start and navigate through courses and learning paths. It includes images, descriptions, and information provided by the course owner, all presented in a sleek format. The new content details page will replace the current course and learning path details page. Course Detail Page Learning Path Page 💡 What you can do now: Mark your calendar for 2 June 2025 and check out the new UI to get familiar with the changes. Share your feedback! We’d love to hear your thoughts once the new UI launches. training@quantexa.com Happy learning! 😊FAQ: Gradle taking too long to index
FAQ relevant for: all Academy versions When opening IntelliJ on the Academy VDIs, dependencies may take too long to load, causing the project to not index properly. This happens because IntelliJ attempts to download dependencies but fails as the VDIs do not have access to the internet. We can stop this happening by putting Gradle in offline mode. To do this you need to do the following: First, click on the Gradle panel near the top of the right sidebar: Then, click on the toggle offline mode button: The button will be a lighter grey colour when the offline mode is on Finally, click the refresh button to restart the indexing: Your project should now index a lot quicker (although it may still take 10-20mins)!FAQ: Academy Documentation Links
Hey all! Here are some useful links for the Academies: ETL Configuring .qmodel files Configuring .qentity files Entities, Start Paths, and traversals Defining Traversals Defining Compounds Defining Elements Quantexa Core Library Core Traversals Core Elements Core Templates Address Business Individual Core Compounds Address Business Individual + Individual Synonyms Core Parsers Address Business If Business parse Business If not Business parse Individual Date to Date parts Entity Resolution Resolver Config Resolution Templates Filtered Compounds Compound Exclusions Network Generation Expansion Steps Scoring (v2.1.8) Severity Tooling Assigning Severity to a Score Configuring Score Descriptions Score Descriptions Score Description Rendering Extracting Values from Configuration files Scorecard Configuration For Comprehensions (Scala Documentation) Project Example - Scoring Best Practices UI Search Configuration Expansion Templates Traversal DSL Functions home page Example Note: The Data Engineering is currently on v2.0.1 of the Quantexa Platform and the Scoring Engineer Academy is currently on v2.1.1, however the closest documentation versions we have are for v2.0.8 and v2.1.8 respectively. These versions should be sufficient for you to complete the Academy! Please comment below if you find any other useful links from the Quantexa Documentation Site so we can add them to the list!
📢New Quantexa Academy Licence (March-May 2025)
The training licence for Academy VDIs has been successfully updated. The new licence will remain valid until June 1, 2025. 🚨 Action Required: Please update your licence on your training VDI to ensure your work is not disrupted, as the current licence expires on the 1st of March 2024. 📌 How to update your licence: You can see how to update your licence using this FAQ. _____________________________________________________________________________________________________________________________ Hope this helps, if so, please mark this as answered. Rupali Pandit - Academy Team Lead 👩💻
