Getting the Most from Q Knowledge Graph Perspectives

A Knowledge Graph typically contains rich information about multiple domains. This can be extremely valuable for insights, but in practice the size and density of data in knowledge graphs can make analysis difficult.  

With Quantexa’s Entity Resolution, you can build a network of linked Entities from a wide range of different documents. For example, we could find business, individual and account Entities across internal customer data, corporate registry information and sanctions lists. These Entities will be linked by different relationships, such as shareholdings, ownership and transactions between accounts. This network can act as a single source of truth for your data, though it is likely to be incredibly large, with millions or billions of nodes and edges. 

For many analytical use cases, creating a Knowledge Graph is a first step in making this data more accessible. A Knowledge Graph can be created from these big networks by treating Entities as nodes, and relationships as edges. Different node and edge types can be used to represent distinct types of Entities and relationships. 

Because a Knowledge Graph represents a full view of data from Entity Resolution, only some nodes and edges may be relevant to specific analysis. For example, transactions may be irrelevant for analyzing circular ownership of businesses. In a similar fashion, shareholding information may be irrelevant for analyzing certain transaction patterns. Exploring the whole Knowledge Graph for either of these use cases can result in inefficient use of compute resources when querying the Graph, and at worst, can result in subtly incorrect results and conclusions. 

To resolve this issue, we can use Q Knowledge Graph Perspectives. A Perspective is a derived view of the original Knowledge Graph, defined in such a way to focus on specific, relevant information for an analytical question or use case. Importantly, we can derive as many Perspectives as there are use cases, each representing the original Knowledge Graph in a way that maintains the most relevant nodes and edges. 

Here we will explore two ways Q Knowledge Graph Perspectives can be used: 

• To filter a Knowledge Graph (keeping only relevant nodes and edges) 

• To augment a Knowledge Graph (deriving implicit edges not originally present) 

Filtering the Graph 

Let’s return to an example where we want to perform analysis of company ownership within our Graph. We can use a Perspective to narrow the Graph down to a subset of important node and relationship types. 

Figure 1. The original Knowledge Graph (left) and an ownership Perspective derived from it (right). Only shareholder edges are kept. Accounts, account ownership, and transactions are dropped, since they are not relevant for this specific analysis.

This has implications for efficiency of Graph querying and analysis. Running our ownership-related queries across this Perspective means that the Graph is smaller and cheaper to traverse, particularly if we don’t define extremely specific node and edge types in our queries. In addition, there is no additional storage cost for this new graph, because Q Knowledge Graph Perspectives reference the original knowledge graph data matrices wherever possible, rather than writing new copies.

Less obviously, but often more importantly, it stops irrelevant structure influencing your results. Certain Graph algorithms will treat different node and relationship types as equal, which may not be desirable if trying to answer specific questions. While deriving Graph structure through Entity resolution of multiple data sources provides a solid data foundation, the calculation of e.g. PageRank to explore shareholding patterns specifically, would be unduly influenced by the presence of other irrelevant data, e.g. payment flows (Fig. 1). Without filtering the Graph for the relevant nodes and edges, PageRank could output misleading results and be used to draw incorrect conclusions. Here, Perspectives help to narrow the focus of graph analytics, to answer contextual questions about your data.

Making implicit relationships explicit

While filtering the graph is useful for many analytical use cases, there are some where the actual relationships and patterns we want to explore are not yet present in a knowledge graph. Rather than filter, we might want to alter the schema of a knowledge graph through derived edges, which has separate implications for scalability and analytical inference.

Figure 2. A payments Perspective. Wherever an Individual owns an Account that transacts with an Account owned by another Individual, a new ‘payed’ relationship is created between the two Individuals. The Account nodes are no longer required and are dropped.

We can use Perspectives to add new edges to a knowledge graph, through augmentation. With augmentation Perspectives, we can specify a schema pattern that we want to replace with another. For example, if we want to explore transactions between Individuals in our knowledge graph, rather than operate on a graph containing account ownership, we can directly augment the graph to create relationships directly between these Individuals, where a payment has occurred (Fig. 2). Q Knowledge Graph achieves these schema alterations through matrix multiplication, making these operations lightning fast when compared to other Graph Database solutions.

There are several advantages when using this Perspective for Individual-based payment analytics. The knowledge graph is now effectively ‘compressed’, containing only relevant nodes and edges for this specific use case. Any graph algorithms running over this augmented Perspective now have fewer ‘hops’ to traverse, reducing their run time and computational cost.

Similarly to the filtering example, there is also an advantage in terms of graph algorithm result interpretability. When looking at payment chains between individuals, we might care less about e.g. how many Accounts an Individual owns, and more about the flow of funds from one individual to others. The removal of irrelevant Account nodes here will prevent confusing results from graph algorithms like centralities and community detection, that are heavily influenced by the structure of a graph.

Building Perspectives: a worked example

In the following worked example, we build a knowledge graph and derive two Perspectives from it.

Creating a Knowledge Graph

Here, our Knowledge Graph has been created through the output of Quantexa’s Batch Resolver entity resolution. It has three Entity types and six relationship types, spanning both ownership and payments.

import qknowledgegraph as qkg

kg = qkg.KnowledgeGraph.load("kg.kginfo", spark)
kg.overview()

Figure 3. The metagraph (knowledge graph schema) of our knowledge graph. Perspectives can be defined by working directly with the metagraph.

In Q Knowledge Graph you define a Perspective by manipulating the metagraph (Fig. 3). You can define a Perspective, and sketch the modification to the metagraph to validate that you have made the intended changes, before running Perspective generation. Once you’re happy, you can apply the Perspective to the knowledge graph and save it to disk.

An ownership Perspective, by filtering

ownership_mg = kg.metagraph.select_triplets(
    "individual-shareholder-business",
    "business-shareholder-business",
)

ownership = kg.save_perspective(
    ownership_mg, name="ownership", output_root=OUTPUT_ROOT,
)

Figure 4. The ownership Perspective metagraph. Faded node and relationship types will be dropped upon running Perspective generation.

A payments Perspective, by augmenting

Here we declare the path we want to collapse, name the relationship it produces, and then keep only Individuals.

from qknowledgegraph.triplet import Triplet

ind_owner_acc = Triplet("individual", "owner", "account")
acc_txn_acc = Triplet("account", "txn", "account")

payments_mg = kg.metagraph.augment(
    ("pays", [ind_owner_acc, acc_txn_acc, ind_owner_acc.invert()])
).select_nodes("individual")

payments = kg.save_perspective(
    payments_mg, name="payments", output_root=OUTPUT_ROOT,
    remove_self_loops=True,
)

Note that remove_self_loops=True makes a difference to the Perspective result. Without it, every Individual who moves money between two of their own Accounts will be represented as paying themselves, which may or may not be desirable for different graph algorithms and analysis.

Figure 5. The payments Perspective metagraph. The dotted pays edge is derived, and everything it was derived from has been dropped.

Perspectives in practice

To demonstrate the impact of using Perspectives, we generated a graph using the same schema as in the examples above. The knowledge graph contains one million nodes and 5.4 million edges, made up of 400,000 Individuals, 100,000 Businesses and 500,000 Accounts. We then build both types of Perspectives mentioned above and run the same analysis on each.

Both the ownership and payments perspectives have drastically different structural properties (Table 1). As well as being smaller, their density and degree distributions are different to the original knowledge graph. It is important to remember that these structural changes will have strong implications for analytics and graph algorithms.

Summary

When working with complex graph data, a good pattern is to create one ‘master’ Knowledge Graph, and derive several Perspectives that each suit a particular set of questions and analyses.

Perspectives help to focus analytics on nodes, edges and patterns in your graph data that are relevant to specific use cases. Using Perspectives smartly can unlock insights that are difficult to spot in rich, dense graphs.