Master Data Management or MDM as it is more commonly known is often considered a difficult to understand concept. Is it a product or an architectural concept? Compounding the problem are the different software makers who claim to provide the MDM solution.
In this article I take you to a back-stage tour of an MDM initiative with a large insurance company based in South Central US, lessons learned and also the tenets which were used as the guiding principles in that project.
What is Master Data Management?
Master Data Management (MDM) is, at its core, reference data – about customers, partners, products and other critical entities – which would be utilized by your organization as a whole. Having this single version of truth ensures consistency in processes by different business units, and better informed business decisions – overall faster responsiveness to change.
Other business drivers for MDM are – synergies for cross-sell and up-sell, improvement in customer satisfaction, easier compliance and regulatory reporting, legacy system integration & augmentation and automation of business processes which eliminate manual work.
From a technical perspective, MDM projects are driven by the requirements for data matching – for example, identify different representations of the same entity (e.g. different addresses for the same Account) across heterogeneous sources.
Other technical requirements for MDM include standardization of data to create a Gold Reference, and enriching the master data with date purchased from third parties. Other technical benefits include centralized management of a unified application landscape with a standard tool set, and universal connectivity between data sources leading to ease of consumption.
At the end of the day, faulty data leads to faulty business decisions. For example, if you do not have standardization of country code and your records include US, USA, America as code for the same country, then it will be impossible to create roll-up reports summarizing a consolidated view of your business.
MDM is not one tool or technology – despite the fact that every vendor claims to offer an MDM tool – nor does it conceptually apply in the same way to different organizations. It is best to approach MDM by looking at your business goals. Then create a list of requirements from your business users and identify issues which may potentially stop your company from realizing its vision.
There do exist common MDM attributes, which you can use as a reference framework for your Master Data Management initiative. Let’s take a closer look.
How is MDM related to Enterprise Architecture?
Someone recently asked me, “How do I implement a MDM project in my company”? My answer was “what problems do you have?” Developing a holistic view of the problems the business users encounter in their daily life; and by investigating the root cause of these problems – we can arrive at the right technical architecture strategies.
Both MDM and Enterprise Architecture share common goals – better business insight, support business growth and reduce cost. Absence of MDM leads to bad data which in turn leads to incorrect business insight. Similarly, lack of enterprise architecture standards leads to reliance on manual processes like spreadsheets which in turn leads to user errors and incorrect business insight.
It is always good to have a holistic enterprise architecture perspective. MDM and enterprise architecture complement each other – and you can think of MDM as a subset of Enterprise Architecture.
Understanding the root cause of issues is the first step in both MDM as well as Enterprise Architecture projects.
An accounting manager may report that he sees a lot of inconsistencies in data. Ask, why? Are these inconsistencies because some of your business processes are spreadsheet based, and each user has his own version of spreadsheet on his laptop?
In another classic case, as project manager, I once had two different business units from one vendor providing me with two different quotes for the same product. Why did that happen? Is it that the vendor did not have a good territory management solution, or is it that they did not have one view of the customer (i.e. me)?
Enterprise Architecture awareness is the first step in planning an MDM initiative and it is always good to roll out your MDM project as part of an overall enterprise architecture strategy.
Trends in MDM: Past, Present, Future
Nobody starts an MDM initiative because it’s cool. MDM becomes inevitable as your organizational processes evolve and mature. Here are some trends in the MDM landscape.
In the beginning
In the 1990s, many companies adopted ERP applications to consolidate silos of information existing in Mainframes, departmental DOS based systems and simple emails. As additional applications – e.g. CRM, Marketing – were adopted in addition to the ERP systems, there arose a need for solutions which maintain data consistency across the organization. ETL tools which extract, transform and then load data into a central database – Data Warehouse (DW) – became popular. Analytic MDM as this came to be known, consists of an enterprise Data Warehouse – or departmental Data Marts – serving as a trusted source of company’s data. This data repository is used for financial reporting and other business intelligence applications. Analytic MDM is characterized by centralized control and DW/BI centric approach.
However, Data Warehouses – analytic MDM – do not improve the quality of data in the heterogeneous applications and operational data sources. As a result, poor quality & inconsistent data finds its way from the operational data source into the Data Warehouse; and results in incorrect reporting and decision making.
So then organizations started looking at their transactional applications. Applications that automate key business processes – e.g. sales, service, accounts payable, manufacturing – rely on data about the objects that are involved in transactions like Customer & Product, as well as transactional information like price and discount. These objects vary depending on the industry, but in general the core operational data revolves around Accounts and Products. Integration technologies which allow operational data to flow in a heterogeneous application landscape became popular, and this led to the advent of SOA and other integration approaches.
Thus, “Operational MDM” consists of solutions which standardize and clean data in these objects which are critical to the functioning of the business. Operational MDM is characterized by engagement of users (through workflows in data change requests) and search before create (e.g. LOV) paradigms.
Today
Today, however the organizational boundaries have become fuzzy with cloud applications. Some of your applications may be fully on-premise, another set of your hybrid on-premise apps may store data in a Storage Cloud while other applications in your environment may be true cloud solutions.
In this world of cloud applications, integration has become vital so that users can have a unified view of their business. For example, if you show sales reps not just a list of opportunities or leads, but all sales to a given customer in the past 30 days, and they’ll sell more effectively. This order history of a customer may reside in another order processing system like SAP and may need to be pulled (i.e. integrated) in the front-facing app like Salesforce Sales Cloud which sales reps use.
However, integration has evolved and become easier. With the advent of simplified integration technologies (REST, OData & others), productized integration (connectors which include prebuilt data mappings, transformations, and process orchestrations) and Cloud Integration-as-a-Service offerings, integration projects are measured in days not months or years. We are no longer at the mercy of all-or-nothing SOA/ESB integrations.
As such Enterprise MDM – which manages both operational and analytic data – is now possible for even mid-size companies with the availability of easy integration solutions. In addition, cloud based data quality services and social stewardship of data facilitate MDM projects. Cleansing of operational data improves the operational efficiencies of the business processes; now the cross-references and hierarchies utilized by the Data Warehouse are true representations of how the business is actually functioning.
In Enterprise MDM business insight through analytical processes are fed back to the operational side of the business – closing the loop between Analytic MDM and Operational MDM. The Enterprise MDM solution addresses data quality issues across your entire application landscape.
Long Term: So what’s next?
In the future we see evolution of MDM initiatives which span multiple domains, can take advantage of Big Data & and link existing customer master data with social networks. A trend we see is mining of Big Data to populate social MDM. We see that systems maintaining the master customer profile data – the system of engagements in which back-office data is surfaced and manipulated – will become a key integration point and centerpiece in the MDM strategy.
We already are seeing the coming together of Analytic and Operational MDM as Enterprise MDM today. In addition, we also see rapid growth of MDM in mid –market as well as across industries. MDM appears to be evolving away from data centric hubs to a shared services architecture (process hubs).
Data Governance & Data Enrichment are close cousins of MDM and we are seeing both these areas gaining huge importance.
How To Analyze Your Current State
1. Review Your Application Landscape
2. Review Your Enterprise Data Architecture Maturity
3. Review Your Data Governance Maturity
1. Review Your Application Landscape
Identify your business applications.
Start by making an inventory of all your applications. Check with business users if they have some departmental applications and software that your global IT team is not aware of.
The Shearing Layer concept – PACE Layering in architectural terms as proposed by Gartner – has emerged as one of the most widely adopted approaches for classifying the application landscape in an organization. In this approach you examine & classify the applications in your environment into different groups based on their need to adapt and change at different speeds; and also based on the nature of their interaction with customers. The different systems in your organization can be grouped as Applications for Engagement (Sales Cloud, Service Cloud) and Back-office Applications (order processing, financial systems and other) and Industry Specific Applications (e.g. Claims database in Insurance vertical).
Classify your business applications.
After you have listed out all your business applications &classify them. In this step you also study the human processes which transcend these applications; and make a list of key use-cases. Discover what are the manual processes at the boundaries of these three groups of applications (e.g. once an opportunity is closed in Sales Cloud then do you need to manually login to another application to initiate order processing?). Are there spreadsheet based applications which lead to data inconsistency and manual work in data entry? Can you separate your business users into discrete groups? What are the common complaints from users in each group?
This examination & classification of your application landscape and the human processes in your organization would bring to fore the nature of application integration needed, issues with data quality and the top requirements of your business users.
2. Review Your Data Maturity
A quick and dirty examination of your organizations data management maturity will give you a perspective into the approaches which you would adopt in your MDM initiative. How do you rank your organization in these following three areas: Platform & Architecture (architecture standards & approach, data integration), Data Quality (data cleaning & data quality strategy) & Data Governance (metadata management)?
In this process, from MDM perspective you first identify all your data sources, who they are used by and how they are used. Then comes the process of classification – you note which are the authoritative sources (trusted system of record) and which are the systems of engagement. Also, note what are the discrepancies in data in different systems; and which entities/objects are candidates for standardization and consolidation.
Based on the above assessment, you will adopt a phased plan for data improvement, a plan which is tailored to your organization’s strengths and priorities.
3. Review Your Data Governance Maturity
According to Wikipedia –“Data governance is a control that ensures that the data entry by an operations team member or by an automated process meets precise standards, such as a business rule, a data definition and data integrity constraints in the data model”.
Data governance in your organization can range from Non-Existent (application centric, meets business needs on project basis) to Basic (policy driven standardization on common technologies across projects), to Advanced (data and metadata shared across sources, integrated view of compliance requirements and formalized organization with roles and responsibilities).
Data Governance Made Easy
Data governance can be as simple as including validation rules in your business applications to prevent users from entering misaligned values – e.g. a specific format for email addresses. We also see other functions which ensure data quality is maintained in mature Cloud applications like Salesforce – page-layouts and record-types to provide different views of the same data for different users. Other examples include use of Picklist to enter reference data (like country codes). Security rules and profiles control who enters what – and since Salesforce tracks audit information (through Field Audit Trail function) you can easily see what changes are made by whom. All these safeguards ensure that the data coming in is clean. We see many organizations leveraging these data validation features and utilizing Salesforce as a trusted system of entry and engagement.
In the data governance framework you also decide which is the system of entry of records. E.g in which system – Salesforce or SAP or Oracle – will new accounts be created?
Using front-office app (e.g. Salesforce) as a System of Entry
If you choose to use a front-facing application like Salesforce as the system of entry then you can disallow the creation of Master Records in your back-office system. Other systems keep their local databases but are prevented from creating new records directly. Records are only created in Salesforce and propagated back through API calls to these local data stores. Channeling all record creation through Salesforce can help to ensure that duplicates are minimized and that there are fewer conflicting versions to be consolidated. With data enrichment services like data.com integrated into Salesforce, cleansing and enrichment can also happen at the point of entry.Using a back-office app (e.g. SAP) as a System of Entry
Alternatively, you could have a policy that master records are not created in Salesforce but in your back office systems. Then in Salesforce you prevent users from creating new values of Master Records (e.g. Accounts) in the user interface. These records could be created in your designated system of records and added to Salesforce through API calls. Also, using next generation tools like Salesforce Canvas, you can embed your back-office system of records (e.g. SAP) in Salesforce. It would appear to your users that they are creating records in one integrated UI (Salesforce) while in reality these records would be created in your back-office (e.g. SAP) and replicated back to Salesforce after they have been validated.A robust data governance framework – and data steward team- is vital to the success of your MDM project. You may need to consider establishing a data governance task force and a workflow based approval process for maintaining trustworthiness of your master data.
Today we see the productizing of data governance frameworks – and applications like Salesforce Cloud provide many out-of-the box capabilities for ensuring upstream data quality which makes data governance easy.
Three Questions to Ask in Planning an MDM Rollout
At this point you have already studied your application landscape and identified which applications are customer facing (applications for engagement). Now with this big picture let’s see what’s involved in a MDM initiative. There are three key questions you need answered first.
1. What is your MDM Architecture Approach?
Gold Standard Vs. Linked Object
Gold Standard
As we have so far examined, MDM in an ideal sense is a master repository which serves as the single source of truth. What constitutes this master data varies, but most commonly included entities are Customer & Product data. The work involved in a MDM project relates to acquisition, transformation and consolidation of master data from heterogeneous sources into a centralized repository. In this traditional world, the MDM data store would be the core repository for all master data objects (Account, Product etc.) and all consumers of master data would make a real-time web service call to that MDM data repository – whenever they needed to enter or work with master data.Thus this Gold Record approach picks the ‘best’ data from each system and creates one master repository as the single source of truth. For example, a customer’s address in this master data repository will be considered as the trusted record, and will be utilized to reconcile any differences in the address for this customer in different applications. There are multiple versions of this single repository approach – create a consolidated gold record by picking out the best records (be ready for data loss) or combine multiple versions into a single record with multiple views (less data loss, but more complex). All in all, this “Gold Standard” approach is still successfully utilized by companies – however the following “Linked Object” approach is becoming more popular as a faster route to MDM.
Linked Approach
A more common approach today is that different applications will have their own local representation of master data objects stored in their own databases. In this environment any application can be both a consumer and a source of master data.We have already discussed how in Enterprise MDM, business insight from Data Warehouses is fed back to the operational data sources making it a closed loop process. In this model, sources will create or update master data in their own local data stores then send that new master data to the MDM system. When new consumers are added to this MDM solution architecture, they replicate data to their local databases. The Linked Object approach keeps all source versions of the objects as they are but ties them together using a shared key.
In the traditional approach, MDM project also includes the development of UIs and APIs through which the data is made available. More commonly however, once MDM project is finished, clean and improved reference data is available, and the different systems and users are naturally integrated with this reference data.
2. Who is your executive sponsor?
Bottom-up approach will NOT work
The number one reasons MDM project managers fail is because they do not recruit senior executive from the business as the champion who is actively interested in and involved in the progress. MDM does not only change how the data is managed but also the business processes and practices across the organizations will need to change. These changes will be disruptive so only a senior exec with enough clout can get you through. Bottom-up approach or simply making it an IT project will never work.
3. What is your business case for MDM?
IT centric approach will NOT work
Executives would be more interested if you position MDM as improving customer communication and reporting, rather than an in infrastructure upgrade project. However you do need IT’s involvement to make sure that your MDM initiative is grounded in reality.
Your executive sponsor and business people would not be interested in the technology improvement even if you portray it as a cost saving measure. Top executives are interested in more than cost saving (unless your company is in serious problem) – they are, or should be, interested in growth. They are interested in solving business problems which inhibit growth.
If you don’t know what the main problems with business are then interview business users and drill down to the root of their problem. Then ask how much it is costing the business. Talk to the business leaders and ask them to spell out their top 3 goals and review the issues in context with these goals.
Promote MDM as an essential, competitive business strategy in which the IT would enable high value info and data to be used repeatedly across the business processes. Your chances of success are highest if your business users agree to begin MDM focused on customer centricity or product/service optimization.
Step-by-Step MDM Implementation Cheatsheet
Should you go in for the Big Bang approach or a phased approach? It is not an either/or scenario. What works best is if you craft an overall strategy and then create a phased roadmap for execution. It is important to have small wins for some near term objectives so your business users (& exec sponsor) remain engaged.
MDM Implementation CheatSheet
Phase 1: Discovery, Analysis & Design
• Ascertain business requirements & examine your current state
• ROI projection.
• SI, Vendor, & Product evaluation
• Plan the MDM architecture – Gold Standard vs Linked Object; and identify which of your applications will function as systems of entry, for data enrichment and trusted system of record?
• Create a cross-functional MDM team and POC if needed
• Capture and model the most essential business processes, data flows; and identify ETL & integration requirements.
• What-If scenarios analysis – Plan for organizational and cultural change, future corporate directions, scalability, potential new markets, M&A, and new or unexpected data sources – Plan for business agility and change.
• Roadmap & 3D rollout plan – which takes into account all the three dimensions – People, Processes and Practices – of your organization. Standardizing master data involved reducing data silos and changes in cross-functional business applications.
Identify which people are impacted by specific aspect of the MDM project and create communication tailored to overcome their resistance.
• Establish a data governance team which would focus on creating and maintaining trustworthy master data. Data governance brings accountability and lays out policies which are important in risk and compliance audits.
• Solidify the metrics for success which serve as ROI indicators. If the purpose of your MDM is to provide reliable customer data to sales teams, then improvement in customer retention or increase in CSAT could serve as metrics.Phase 2:
• Limited scope deployment with a single department with one entity (e.g. Customer/Account).
Phase 3-a:
• Rollout enterprise wide deployment with one entity (e.g. Account).
Phase 3-b:
• Roll out enterprise wide deployment with more than one entity (e.g. Customer, Partners Product, Call Center)
MDM Tools & Technologies
What are the high level criteria for choosing MDM and Data Governance tools?
According to The MDM Institute the following are the key attributes of any MDM technology: Security (multi-level role based security), identity management, rules and policy management (workflows), global search, data model and business glossary management, data cleaning and enrichment, ease of integration, reporting and analytics, Cloud and Big Data, and support from leading SIs.
Here are some of the tools you would need in your arsenal:
Security review tool, ETL & Integration tools, Data Enrichment, Automation apps, De-Duplication tools
Five MDM Landmines
1. Not having an executive sponsor, using a bottom up approach in partnership with middle-managers who don’t have enough clout
2. Not creating a business case and focussing on technology improvement
3. Starting with an all-or nothing approach instead of a phased rollout
4. Not leveraging next generation Cloud and data enrichment applications
5. Not identifying metrics to make MDM a measurable process
Anatomy of a Successful Project
MDM Case Study
This case study is based on a project with a large insurance company I worked with in central US. In the requirements gathering phase for this limited scope MDM initiative, we discovered that the customer had numerous challenges in their business.
Business users complained that reports were error prone and data was generally inconsistent. There was a lot of duplication of data. Much of the data was outdated. In addition, the business users did not have a unified view of business. The sales people needed to log into multiple systems to view past orders and also to generate orders for a customer.
An example of data inconsistency was multiple country codes (UK, GB, US, USA, In, India etc.) as a result of which reports and dashboards were error prone. There was issue with marketing data as well. Leads were coming in from marketing but were not getting assigned correctly. Top management did not have a global view of business as account hierarchies were either missing or not accurate.
In this project we followed a phased approach and a Linked Object MDM model. After key business processes and data sources (e.g. Salesforce Sales Cloud, Order Processing systems, Call Center Application) were identified, we created an integration framework which pulled in data from on-premise systems to the Salesforce Cloud – at the same time, transforming and cleaning it using ETL processes.
Hand in hand with the integration phase was the consolidation work which involved putting in place business rules as data was brought in from different systems in Salesforce Sales Cloud. This step also included building hierarchies so that the sales management can view the global customer account and yet drill down into the regional accounts – for example, the geographically distributed offices of ABC Corp roll up into one global ABC Corp HQ account in the Sales Cloud. This was possible using the organizational hierarchy information from the data enrichment service data.com.
data.com enrichment engine also provided matching services and a matching algorithm using which duplicate data in multiple repositories was compared and cleaned. Since we decided to use Salesforce as our system of entry – and it was integrated with data.com – cleansing and enrichment happened at the point of entry ensuring that data coming in was clean. Data quality checks were put in place using data.com to ensure that data once cleaned remained clean.
As a best practice it is good to minimize the systems of entry to master data – multiple sources of data lead to complexity and need more time and effort to manage. As we minimized the systems of entry, we had to plan for integrations and changes in our business processes through which new records can created. For example, instead of manually generating an order in the back-office system, an integration of Sales Cloud with your back-office order processing system triggers an order generation when an opportunity is marked as closed in Sales Cloud. The account information is passed automatically to the order processing system. Master entities like accounts will be created and updated manually in the customer facing Sales Cloud only – the users in other applications have a Salesforce widget in the screen of these applications to allow creating of these master entities from one browser window.
Standardization of sales and marketing data and identification of duplicates resolved many of the issues with lead assignment and incorrect reporting due to data inconsistencies. Data duplication was addressed with the help of the new Duplicate Management function in Salesforce.
As we see in this project – any initiative is driven by the business goals. In this case the goal was having a unified view of the customer and better business insight for management. Other MDM projects may have a much broader scope.
However all MDM projects revolve around improving the key organization data entities through different means – cleaning, enrichment, de-duplication, standardization. While this makes MDM seem like a data centric project, it is actually a business centric initiative – always. In every MDM project the end goal is not data. The end goal is to have more efficient and agile business processes, better business insight and happier customers.
Summary
MDM (Master Data Management) in a Salesforce cloud environment can be summarized in the following three steps:
- Identify the key entities in the different databases in your organization. These could be Account, Contact, Product etc.
- Examine if you would like Salesforce to function as the system of records or if you would like to have a separate MDM hub?
- Establish governance. As an example, if Salesforce is the system of record for Contacts, you can prevent creation of contacts in other databases and require that Salesforce function as the system of truth for the Contact entity. You can also enforce validation rules and Picklists so that the data is entered in the correct format ensuring correctness of information.
The above three steps give a very simplified view of how to craft an MDM strategy.
MDM is a very confusing topic for most architects and managers. But in reality it is all common sense business process rationalization – align your applications with your business goals and everything falls in place. The purpose of this article is to go beyond the marketing jargon and share what real world MDM looks like.
I would welcome your tweets if you enjoyed this article!