FAQ - List of Academy Frequently Asked Questions
Below is a list of FAQ's for the Quantexa Academies. Have an idea for an FAQ? Please let us know by emailing training@quantexa.com with the idea or by commenting on this post here! General Quantexa Documentation Links for the Academy Gradle taking too long to index I tried to run a spark shell script and got a "Permission denied" error message My Quantexa Licence has expired, how do I update it? I'm trying to run a script and seeing an error about "Unrecognized option: -s" Fusion What are Document Attributes used for? What are Entity Attributes used for? What are (Entity) Records? What are Traversals? Elastic Elasticsearch "cluster health: Not connected" in Chrome / "Connection refused" when running Load script error I'm missing data in Elasticsearch / my number of docs are wrong Resolver Config Why are search constraints not working & how to use wildcards (* characters) within field sets? How can I configure the timeline? Batch Resolver (ENG) I tried to run Batch Resolver (ENG) but got an error about Timestamps / Dates Scoring My score isn't triggering / doesn't appear in the UI How can I handle dates when writing a Quantexa Score in Scala? Checking Score Outputs (Optional Sink Step) UI I tried to start my UI but some of the apps didn't start up / I got a UI error Search configuration could not be loaded in the UI I can't find Addresses or Individuals when I search in my UI! Data / Attributes aren't showing up in my UI! VDI VDI Usage and Common Issues I'm having trouble pasting into my VDI Admin I can't access a website or course BA Academy List of Technical Business Analyst (BA) Academy Frequently Asked Questions Don't forget to bookmark this page in the upper right corner to save it for easy access.FAQ: 💻️ VDI Usage and Common Issues
FAQ relevant for: Academies that require a VDI Your training VDI can be accessed via this link. Before it can be used however, you will need to start the machine that it runs on and you also should stop the machine after you have finished using it. The VDI can be started and stopped via this page on the Quantexa Community, and there is also a refresh button to check the status of the VDI. After you start the VDI, you will need to wait a few minutes for it to boot up before it will be accessible. Common VDI issues & error messages: “You have been disconnected.” - This message will appear when the VDI hasn’t been started yet. To solve it you just need to start the VDI and wait a few minutes. “An internal error has occured within the Guacamole server, and the connection has been terminated. If the problem persists, please notify your system administrator, or check your system logs.” - This message will sometimes appear if you have started your VDI but then tried to access it before it has fully booted up. If you want a few more minutes then this should disappear and your VDI should be useable. If you still can't access your VDI after 10-15 minutes, it's likely due to your organization's firewall. “The Guacamole server is denying access to this connection because you have exhausted the limit for simultaneous connection use by an individual user. Please close one or more connections and try again.” - This message will appear if you try to access the VDI from more than one browser tab at once, but also sometimes can appear at random (possibly caused by network issues). To solve it, close all tabs then wait a second before trying to open a fresh VDI tab. Alternatively you can hit the “Logout” button then on the next page the “Re-login” button and this usually fixes it too. Poor network conditions may cause this problem to persist. "The remote desktop server is currently unreachable. If the problem persists, please notify your system administrator, or check your system logs." - This error message means there is something wrong with the boot disk of the VDI and will need to be resolved by our cloud team. In this instance, please reach out to an ATL or create a new post in the Academy Q&A topic and we will get the team to resolve it for you. If any of the above problems persist and you are unable to solve them, then please reach out to the training team via the Academy Q&A Topic.📢 Welcome! Please Read Before Posting
Hey there, and welcome to the Academy Discussion space of the Quantexa Community! 😀 The purpose of this space is for people to be able to get help and information as they journey through the Quantexa Academy! Encountered a bug which you can't see how to fix? Have a query about a specific area of the Academy task? Need help with understanding some of the key concepts? Let us know by asking a question! Before posting a question however, please review the following article and video around debugging and getting the most out of the Community, which you can find here: https://community.quantexa.com/kb/articles/264-1-how-to-maximize-value-from-community-support Remember, vague discussions are hard to act on so please provide as much detail as you can! If you have any questions about access issues (e.g. you don't have access to LearnUpon or the Quantexa Docs site) then please email training@quantexa.com rather than posting here, thanks! 😀FAQ: I'm missing data in Elasticsearch / my number of docs are wrong
FAQ relevant for: all Academy versions If you have completed the ETL pipeline stages of your project and uploaded the data to ElasticSearch, then when checking your indices in the ElasticSearch Head plugin on Chrome you should have numbers similar to the picture below (to get a bigger version of the image, right click it and chose the option to open it in a new tab). Note: If your numbers vary a little bit from these, for example having 152k address instead of 148k, then that's ok - the numbers will change a little for the resolver indices (Individual/Address/Business) based on the compound keys you have imported in the respective *.qentity Fusion config files. If your numbers are significantly different to this, then you will want to go back through your ETL pipeline and carefully check each stage to see if there is somewhere that you lose the data along the way. A good way to approach this problem is to work forwards from CreateCaseClass and check the output of each stage to find the problem area. You should also use the counts in ElasticSearch to guide you - for example if you have only half the number of businesses listed above, and no individuals, it lets you know that you probably haven't joined your Third Parties onto the ICIJ document properly, and so you would want to go back and double check how you have done this join and on what fields. Specific points to consider: Have I correctly parsed all of the necessary fields in my qmodel files? Have I used the correct type of joins in CreateCaseClass, and have I joined on the correct fields? Have I outputted the correct Dataset at the end of CreateCaseClass? Have I loaded up the DocumentDataModel.parquet (the output of CreateCaseClass) into a Spark-Shell to check the output there? Have I correctly identified and defined all relevant start paths in my qentity files? Do I have a good range of compound keys for each Entity? If you are convinced that you have done all of the above correctly then you can try to clear the data from ElasticSearch, restart the service and then re-upload the data to Elastic using the following three commands: curl -X DELETE 'http://localhost:9200/_all' sudo systemctl restart elasticsearch.service ./runQSS.sh -s com.quantexa.academy.task.icij.model.etl.IcijLoadElasticScript -c ../external.conf -r elastic.icijAcademy Training VDIs 💻️ - A new way to start and stop your VDI!
Hey all, This is an announcement to say we are launching a new way to start and stop your Academy training VDI! Currently this is done by using the Slackbot on our Academy Slack channel, however going forward from today, you should stop using Slack and instead use the new tool on the Quantexa Community, which you can find at this link (we advise you bookmark this): https://community.quantexa.com/categories/academy-vdi This new tool on the Community works in a similar way to how the Slackbot works: there are buttons to Start and Stop the VDI, with a refresh button that replaces the "/vdi status" command. After you click the Start button to start the VDI, you will need to wait a few minutes for the VDI to boot up before it is available to use, exactly the same as how it was when using the old Slack method. The website that you access your VDI from remains unchanged: https://training.quantexa.com/vdi/#/ We also have a new FAQ coving some common issues encountered when using the VDIs: Please reach out to the training team if you have any issues with the new VDI tool. We will be decommisioning the Slack channel in the near future.FAQ: I can't find Addresses or Individuals when I search in my UI!
FAQ relevant for: all Academy versions If you perform a search in your UI and it only returns eg. Business entities but not Individuals or Addresses, then there are a few things you can check to resolve this issue. Firstly, remember that when you perform a general search (red box below), it will search within any field sets (orange box) you have configured in the search definitions of the resolver config. If you have configured these search definitions for all entity types, then also double check for any spelling errors in the resolver config in terms of fields that you are pointing at in the underlying data - these searches go off of fields in the cleansed Document Data Model. The other place to check is that your elastic has the data you expect in it - do you have address and individual data there? If not, then you may want to review your ETL pipeline, and check out this related FAQ. Lastly - you may need to refresh the security permissions to be able to access these additional Entities. You can do by running: ./drop_recreate_databases.sh In your home directory on the VDI (while the UI isn't running). After this, you should see the additional search fields and be able to find these entities. Let us know in a new post if you are still having issues after performing these checks!📢 Introducing the Community Training Program 📢
Education Services and the Quantexa Community are excited to announce the launch of the Community Training Program: Quantexa User Foundations and Quantexa Technical Foundations! Powered by the Quantexa Academy, the Community Training Program provides registered Customer and Partner Community Members an opportunity to access the content of the Quantexa User Foundations and Quantexa Technical Foundations programs easily and at no extra cost. "What I love most about the Community Training Program is the speed of providing access to training materials. This has been particularly useful to client Product Managers and business stakeholders being onboarded onto projects. They can jump right into the topics that interest them. It’s a flexible and accessible approach to learning that is responsive to individual needs." , Technology Account Partner Benefits for You and Your Team Reference & Refresher Material: Perfect for revisiting key concepts and keeping skills sharp, especially for those who have previously completed the Academy. Enhanced Knowledge: A great way for non-end users to build awareness of how their colleagues use the Platform. Team Development: Share this with your team to expand their knowledge of Quantexa’s functionalities— they may uncover new features or techniques to enhance their work. Quantexa User Foundations (QUF) This program introduces the key concepts of the Quantexa Platform through tutorials, demos, and easy-to-follow guides for end-users, such as Analysts and Investigators. Modules: Introduction to Quantexa Decision Intelligence Platform: A high-level introduction to what the Quantexa Platform is and what it does. The module also explains some of Quantexa's capabilities and the benefits it offers to customers looking for a cutting-edge and user-friendly Decision Intelligence Platform. Core Concepts: This module introduces the foundational concepts of the Quantexa Platform, including Networks, Documents, and Entities, while exploring features like Search, Investigations, and Scoring to help users understand data structure, Entity Resolution, and Network creation. Search: This module provides an in-depth overview of the Quantexa Platform’s Search function, teaching users how to configure Advanced Search Options, construct effective queries, and interpret Search Results to support investigative work. Investigations: This module introduces the Quantexa Platform’s User Interface for conducting Investigations, focusing on mapping the 5W process, navigating Network Visualizations, and utilizing features like the Multi-Panel UI, Contextual Search, Scoring Panel, and Entity Lab. Quantexa Technical Foundations (QTF) This program introduces some of the more technical components of our Platform, namely Data Fusion (ETL) and Assess (Scoring). We recommend completing Quantexa User Foundations (QUF) before beginning Quantexa Technical Foundations (QTF). Modules: ETL and Data Fusion: The module covers the key concepts and definitions related to the Extract, Transform, and Load (ETL) process utilizing Data Fusion: Quantexa’s data modeling tool. Scoring with Assess: This module introduces you to the key concepts and functionalities of Assess: Quantexa’s Modelling and Scoring capability. The important process of Scoring, which allows you to identify the subjects of interest linked to your data, will be explained in detail. Learn more: More information on the Quantexa Academy and Community Training Program can be found in the Community Library: 6. Community Training Program - Quantexa Community The Community Training Program is powered by the Quantexa Academy. The materials provide an exciting opportunity for Community Members to learn more about Quantexa's capabilities. Use the materials as a reference or to refresh your knowledge. Program scope The Community Training Program is a work in progress and it… Important Note: The Community Training Program is a great learning opportunity. However, using the materials on the Quantexa Community does not provide an official accreditation. If you require official accreditation, you will need to complete an official certification program through the Quantexa Academy.FAQ: I'm trying to run a script and seeing an error about "Unrecognized option: -s"
FAQ relevant for: all Academy versions If you are trying to run a Scala script (e.g. ImportRawToParqet) and you see the above mentioned error, the main cause of this is that the program can't find the relevant JAR file to run. You may also get an error saying something like " .jar not found" or "jar does not exist, skipping." The solution to this error is to build the relevant JAR file(s) as specified in the Spark Shell script that you tried to run the Scala file with. For example, in the "runQSS.sh" script of the academy task project, you will see that this requires two JARs: the data-source-all Project and Dependency shadow JARs. By building these two JARs and verifying that they are in the correct location and have the correct name to match the full file paths in the relevant Spark Shell script, this should fix the issue. Let us know in a new post if the above solution didn't fix the issue for you!
FAQ: How can I handle dates when writing a Quantexa Score in Scala?
Handling Temporal Data Temporal data refers to any data that is associated with a specific point or period in time. This includes the use of dates, times, intervals or timestamps. This information is useful for indicating when an event has occurred. LocalDate One common way of handling temporal data such as dates in your score logic is by using the LocalDate option. This is part of the java.time package. Using LocalDate allows you to manipulate dates, calculate differences between two dates, format dates into strings and parse strings into Date objects. In Scala you can also use the java.time.format.DateTimeFormatter class to format and parse temporal data into specific patterns. Here is an example below: val dateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd") The above line will help parse your dates into an appropriate format. LocalDate.parse(yourDate, dateTimeFormatter) The above line ensures that the date is parsed as a LocalDate object and would then look like this: "2024-04-17" Now that we have our localDate we can now perform operations which can be useful when writing your score logic. Commonly Used Operations Comparing dates yourDate.isAfter(anotherLocalDate) yourDate.isBefore(anotherLocalDate) yourDate.isEqual(anotherLocalDate) Extracting the month, year and day yourDate.getMonth() yourDate.getYear() yourDate.getDayOfMonth() Calculating the period between two dates To compare two LocalDates we can use the .between method after having imported the java.time.Period class. We can implement the functionality seen above to extract the period and years in the following example: val duration = Period.between(yourDate, anotherLocalDate) duration.getYears() FAQ: Type Mismatch Errors A common issue experienced in the academy is a type mismatch error when using .addRelatedDate. When using .addRelatedDate for the addition of a dynamic date field to your score, it is recommended that you use a Date type. LocalDate can still however be used within your comparison logic.
How to get 100% in the Scoring Academy 🎉
Congratulations to Ahmed Nader () on achieving 100% within the Scoring Academy 🎉 This achievement shows unwavering determination and ambition from Ahmed. Having passed with a perfect result, Ahmed has shared his top tips to help you achieve the same, read them all in our Community Library article: Celebrating a perfect score in the Scoring Academy - Quantexa Community Congratulations to Ahmed Nader from Data Science for passing the Scoring Academy with a perfect result. We want to congratulate Ahmed Nader from Data Science who recently attained the exceptional feat of achieving 100% within the Scoring Academy. This achievement shows unwavering determination and ambition from Ahmed.… These tips are applicable to the Data Engineer Academy and the Scoring Academy. To learn more about the Academy and courses on offer, read our Introduction to Academy article.
