Transaction Explorer not returning results in results finder based on security policy configuration

Hi,

Background Info/Description:

We have transaction explorer configured within our project with specific users, roles and security policies. These roles have access to specific policies that have been configured within our security-bootstrap.json file, allowing them to be able to view data on documents, document attributes, entity attributes, edge attributes and foreign documents based on satisfying the data condition within the expression specified in the templates.

An example of a foreign document template is shown below:

{
            "name": "POBA0_FOREIGN_DOCUMENT_transaction",
            "category":"FOREIGN_DOCUMENT",
            "templateType":"transaction",
            "rules":[
                "*"
            ],
            "expression": "(fromDataGroup == 'POBA' && fromConfidentialityMark == '0') || (toDataGroup == 'POBA' && toConfidentialityMark == '0')"
        }

This is then passed into a policy as shown below:

{
            "name": "POBA0",
            "templates": [
                "POBA0_DOCUMENT_customer",
                "POBA0_DOCUMENT_ATTR_customer",
                "POBA0_RECORD_ATTR_customer-business",
                "POBA0_RECORD_ATTR_customer-email",
                "POBA0_RECORD_ATTR_customer-individual",
                "POBA0_RECORD_ATTR_customer-address",
                "POBA0_RECORD_ATTR_customer-telephone",
                "POBA0_RECORD_ATTR_customer-account",
                "POBA0_DOCUMENT_aggregatedtransaction",
                "POBA0_DOCUMENT_ATTR_aggregatedtransaction",
                "POBA0_RECORD_ATTR_aggregatedtransaction-business",
                "POBA0_RECORD_ATTR_aggregatedtransaction-individual",
                "POBA0_RECORD_ATTR_aggregatedtransaction-address",
                "POBA0_RECORD_ATTR_aggregatedtransaction-account",
                "POBA0_ENTITY_ATTR_business",
                "POBA0_ENTITY_ATTR_email",
                "POBA0_ENTITY_ATTR_individual",
                "POBA0_ENTITY_ATTR_address",
                "POBA0_ENTITY_ATTR_telephone",
                "POBA0_ENTITY_ATTR_account",
                "POBA0_EDGE_ATTR_business",
                "POBA0_EDGE_ATTR_email",
                "POBA0_EDGE_ATTR_individual",
                "POBA0_EDGE_ATTR_address",
                "POBA0_EDGE_ATTR_telephone",
                "POBA0_EDGE_ATTR_account",
                "POBA0_FOREIGN_DOCUMENT_transaction"
            ]
        }

Which is then provided as a policy for the specific role assigned to the user:

{
            "name": "ROLE_FCP-USER",
            "policies": [
                "DABA0",
                "SABA0",
                "FOBA0",
                "NOBA0",
                "POBA0",
                "NIBA0"
            ]
        }

After re-bootstrapping our security tables, I can see that the correct roles are active for the correct users with the expressions correctly applied as shown below for the foreign transactions doc used in TE:

Issue:

The problem we have lies with the data that's showing up within TE. Even though for the foreign document (which are transactions in our case) we have set this rule:

(fromDataGroup == 'POBA' && fromConfidentialityMark == '0') || (toDataGroup == 'POBA' && toConfidentialityMark == '0')

I have noticed that when either fromDataGroup & fromConfidentialityMark are empty (no data coming available) & toDataGroup & toConfidentialityMark have data that matches the above expression criteria, these records are not showing up in the results finder even though they should be. This is also a problem vice versa when toDataGroup & toConfidentialityMark are empty and the from fields have data that matches the criteria. The data that is showing up in the results finder is the one that has all 4 fields populated (but not all of them either, more on that below).

Below is a screenshot showcasing what a user without restriction sees in the results finder when the criteria is met:

Now you will see what the user with the security policies applied sees - note that the results finder is saying it's returning 219,947 results, yet in the viewer we're only seeing the ones that have all 4 fields populated (only 7 records) and none of the others where 2 of the fields have missing data:

I've noticed that this only pulls up the records UNTIL we hit a record with missing data in 2 of the fields, so technically it's not even showing all the data that have all 4 fields populated also.

Does anyone understand why this is happening in this way? Is there some sort of misconfiguration on our end? The exact same expressions were used on V2.3 within the client on a different use case however that was using security V1 & since we're on V2.7.1 we are using security V2.

If anyone could shed some light on this issue that would be very much appreciated!

Support Type: Advice

Version: 2.7.1

Components: Security policies & Transaction Explorer

Debugging: Steps shown above in screenshots, difficult to debug code wise as everything seems to be setup as expected following the docs (Security Bootstrap configuration reference – Quantexa documentation)