Everything Changes, Nothing Stands Still
Heraclitus once said that “All things pass and nothing stays”. If we deny change, there is no growth and stagnation can result, and for this reason to be competitive companies constantly strive to adapt to changing requirements.
The amount of change we have seen in the UK and Europe in the last few years has been unprecedented. Brexit, followed quickly by a global pandemic, accelerated changes in the workplace, with companies rushing to deliver remote working models and digitization of services as consumers required remote access to these services. More recently we‘ve seen the invasion of Ukraine by Russia, fuel and energy prices spiraling up for individuals and organizations. Most consumers are seeing a financial squeeze as inflation ‘bites’ and those changing ‘purchasing’ habits are already being reported by retailers. At the other end of the spectrum, for those people that do have money to invest, we’re seeing a rapid shift in investment patterns, people are taking an organization's ESG performance indicators into account, and companies that are socially and environmentally more responsible are significantly more investable.
All businesses are having to adjust rapidly to the changing environment, in most cases not just to stop stagnation, but to survive.
If we take a technical perspective, having a dynamic business environment drives the need to be responsive and modernize the supporting technical services and applications. As mentioned, recently there has been an acceleration of digitalization, in addition, many companies are still transitioning to the cloud and in doing so they are aiming to consolidate and modernize their technology landscape.
Telecommunication Environment
If we focus on the Telecoms industry, their ideal IT stack looks like the diagram below.
But this is extremely simplistic, as an industry most Telcos have grown through acquisition, so rather than one CRM supporting Billing, Fulfillment and Assurance, you are likely to have numerous systems. There is also the aspect of multiple product offerings, the Telecoms industry has evolved rapidly with Landline, Mobile, Broadband, Media Services, recent fiber rollout and 5G, all with different provisioning, inventory, CRM systems and supporting OSS systems. Just as an example, with potentially multiple CRM systems, customer data is inevitably spread across multiple systems and it is a struggle for many Telco organizations to create a holistic view of a customer that may have multiple products, or where you have an enterprise customer with a complex organizational hierarchy with many subsidiaries, the ability to create a complete view of all interactions with that customer is challenging.
When modernizing systems or migrating solutions to the cloud, organizations have historically struggled with the ability to merge the data from these multiple sources. To the uninitiated, a quick ‘rehosting’ or a ‘lift & shift’ solution may seem attractive, especially if the senior executives are driving a cloud agenda or there are expiring data center equipment leases, aging infrastructure or license renewal deadlines. The benefit, if any, of this ‘lift and shift’ approach is short-lived, you are literally re-platforming your problems, as well as paying for a lot of ‘baggage’ you probably don’t need and ultimately not delivering an improved service. Modernizing your application and systems is of limited value unless you modernize and optimize the target environment and in parallel improve the data on which the new system will run.
An Opportunity to Improve
A system modernization or migration to cloud initiative is an opportunity to improve the solution, provide better services and address any issues you may have with the underlying data, whether this is missing values, non-standardized data or an inability to link data across multiple systems to create a single view of the customer, supplier or other entities in your organization's data. This is balanced against the challenge when performing the migration of potentially disrupting customer services or internal business services. This is why most organizations will ideally look to achieve a phased or iterative approach to their migration or modernization initiative. If for example, we look at customer data, you might need to roll out by customer segment or geography or start with simple individual consumers before addressing the potential complexity of hierarchies in enterprise business customers.
Focusing on the data, specifically customer data, one thing that’s often overlooked when planning for a migration or legacy system replacement is to create a single view of the data. When organizations undertake a migration or modernization, they tend to focus on the mechanics of putting in a new service and generally ignore the important fact that they are likely to be sourcing data from multiple systems that have evolved independently over time. The result is that they will link this data in an ad-hoc way, producing new, or replicating existing problems of having a joined-up view of the customer in the new system.
The Migration Approach
Creating a single customer view as part of the migration or modernization process allows you to bring together everything that you know about a given customer. This dataset of ‘golden‘ records will not only improve the new system but there are additional benefits to consider:
- Creating this customer entity data set means that you are de-duplicating and refining this data, and the data volume being migrated is less, why pay for redundant data or ‘baggage’, if you are paying for data storage in your target environment?
- The other aspect to consider is the other services that this single view of the customer might improve. If we are modernizing our CRM system the single view of data we are creating could also improve, as an example, Sales with Up Sell and Cross Sell opportunities, as well as providing an excellent data set for the Data Science team to support their AI and ML initiatives.
If we look at a traditional data migration approach, there are generally seven phases.
As part of the early phases, 1-3, of a migration or modernization initiative, you need to create a single view based on the data in your existing applications, as part of those early phases you should:
- Combine your data from all sources.
- Analyze migration priorities using a combined view of real data.
- Create candidate migration profiles based on the combined data.
- Apply logic to determine: The parts that are more suited to automated migration – compared to the parts that are incomplete and need attention.
- Look for Conflicts that require having operations teams or the customer double-check their information. Add any missing data to the customer profile.
This approach enables you to address entities such as customer data details before you migrate them. By taking these steps, you simplify the ability to make decisions about your data, operationally create a single view, and enable the migration for a smoother transition.
Using Modern Technologies for Entity Resolution
Technologies now exist that are significantly more advanced, accurate and efficient than the traditional ‘black box’ profiling and data quality tools that companies have traditionally used. Today, technologies exist that take a schema-less approach, sourcing data both from internal systems and external systems. This data can then be parsed, cleansed, and standardized at scale using AI/ML models and then matched with significantly higher levels of accuracy than was previously available. The resulting entities can then be persisted in a data hub for migration, creating a single view of the data in a matter of days or weeks.
It is important to understand the paradigm shift in matching technologies over the last few years. Traditional data matching used record linking, it’s been around for decades and offers a basic approach to resolving entities. The accuracy achieved by these products is significantly lower than true dynamic entity resolution.
By using Dynamic Entity Resolution, organizations can connect all the data sources around customers, addresses, products, and devices – even when the data quality is poor. In addition, linking these customer entities with network analysis allows you to then identify which data you can push into the new application and which data potentially needs manual review.
Quantexa’s Value for Modernization and Migration
Quantexa delivers best-in-class dynamic entity resolution that allows organizations to ingest data from any source system and create a single view of data quickly, be it Customer, Device, Product or Supplier even where there is an issue of poor data quality in the source systems. Paradoxically, the more data the better – in the world of Entity Resolution, more data solves poor data. The advanced functionality available in the Quantexa platform especially the dynamic entity resolution resolves data discrepancies, providing the ability to create profiles around entity attributes based on merging profiles into a trusted ‘golden’ record.
Combining Entity Resolution, and network generation in Quantexa provides the ideal solution for querying across multiple source application records to obtain relevant information. The network generation capability delivers a three-dimensional view of your data to reveal previously hidden relationships in your data, enabling you to filter out irrelevant and unreliable information.
The benefits of using a platform, such as Quantexa, to support your migration or modernization initiative are:
- Improved User Adoption: The number one reason for migration failure in the past, especially when looking at application migrations such as CRM systems, that are focused on entities such as customers, has been due to poor data. The importance of clean, accurate data is often overlooked and impacts user adoption. Creating a single view of the entities as part of the migration or modernization process allows you to bring together everything that you know about a given entity. If, for example, it is a CRM system we are migrating, and the entity is customer, supplier, address, or device. Having data that a user can trust, a holistic view of these entities that allows a user of the system to engage in an informed manner, enabling the CRM users to understand relationships and behavior, prioritize leads and provide a differentiated experience to the customer thus reducing churn. All of these benefits, will accelerate the adoption of the new system.
- Reduced Risk: In 2019 McKinsey found that 75% of cloud migrations ran over budget and 37% ran behind schedule. A lot of the problems associated with a system migration or modernization initiative usually occur very late in a project’s lifecycle. A common error has been to focus on the mechanics of migration only to find that the data you’ve migrated is not ‘fit for purpose’ in the new system, inevitably leading to project costs and timelines spiraling out of control. Adding extra budget up-front to address the data aspect of the migration, to some extent separating the data from the mechanics of the migration and the target system, enables you to then focus on combining data from all sources, analyzing this data, creating migration profiles and address data issues and conflicts in the early phases of the project. This will ultimately reduce the risk of any ‘surprises’ derailing the migration initiative late in the project life cycle.
- Increased Speed to Market: As highlighted earlier organizations embarking on a migration or modernization exercise are not just embarking on this project ‘for the fun of it’. In the dynamic and challenging business environment we live in, there is a business driver, whether it's reducing cost, increasing efficiency, or introducing a new product or service. Using an advanced, automated and accurate approach that incorporates dynamic entity resolution and network generation will enable you to deliver on or ahead of schedule.
- Broader Data Usage and Value: The data that is sourced from multiple systems, curated, and then linked to create a single view of the entity, is not just the foundation for the migrated system, it can also be utilized by other initiatives, whether this is analytic and data science teams, or the foundation for an MDM initiative.
