MDS Versions, Flags and Subscriptions

Recently I collaborated with Andrew Milner in a scenario in MDS SQL 2012 to provide the client the ability to with between multiple Versions and Flags via Subscription Views. Vishal Singh did an excellent job in implementing this scenario in MDS.

Versions and Flags

MDS Model defines the structure of the Data and Version defines the content of the data. In Master Data Services, multiple versions of the master data can be created within a model. Versions created in model can be in sequence fashion (one version after another) or in simultaneous fashion (multiple versions in parallel). This way we ensure we have a dedicated/consistent version of our master data for downstream consumption while at the same time having another version for adding/deleting/modifying members or for testing our model data, without compromising consistency in master data availability.

For example: We may want to have one version of the data marked as “current”, and another set marked as “proposed” using Version Flags. When the proposed version is confirmed as valid and clean, we can move that from proposed to current. So the version marked as “current” will always be the “Golden Copy of the Data”

When to Use Versions

Use versions to:

  • Prevent users from making changes while all data validates successfully against business rules.
  • Lock down a model for use by subscribing systems.

Version Flags

When a version is ready for users or for a subscribing system, we can set a flag to identify the version. We can move flag from version to version as required. Flags help user and downstream applications identify which version of a model to use.

Scenario:

Create Model and 3 Versions (one in Commit Status and other two in Open Status).

An initial version is created by default when a model is created. In order to create two other versions in Open status, commit default version and create two versions from the committed version. Name two versions as PROD and TEST and map it to two different Flags PROD and TEST respectively. Data inserted in two different versions are mutually exclusive meaning they are not related to each other. Data modified in one of the version will not be reflected in another version. Subscription views created based on Version will also have data related to that Version only. Flags help user and downstream applications identify which version of a model to use.

For example: Created a sample Model called “Employee” and created three versions VERSION_1 (committed status), MDS (open status) and SSIS (open status). In the same way created two flags MDS and SSIS mapped to MDS and SSIS versions respectively. Data present in different versions (MDS and SSIS) are not related and thus not in synch.

In Employee entity there is one employee which works in two different technologies MDS and SSIS as highlighted below. Employee (Employee ID: IW003) works in both SSIS and MDS technology but there is a difference in their name in different versions.

mds1

mds2

Records highlighted above are common in MDS and SSIS versions of Employee Entity with only difference in their Name (EmployeeID: IW003) . Subscription view created on the MDS (linked to MDS flag) and SSIS (linked to SSIS flag) versions will have different set of data. Please find below the data present in subscription-view:

mds2sv

Downstream application will fetch data based on the subscription views which they are consuming. If downstream application is consuming MDS version (mapped to MDS flag) will have different data and one consuming SSIS version (mapped to SSIS flag) will have different of data.

Architecture in MDS Acme

We have used below logic in MDS Acme:

mds3arch

In MDS Acme we have two different data sources Production and Test. Data coming from different data sources are getting inserted in the same model (Sample) and same entities. Two versions V1 and V2 are created pointing to Production and Test data sources. Versions V1 and V2 are mapped to Flags F1 and F2 respectively. Subscription views created are linked to Version flags. Downstream applications are consuming data based on the subscription view published. When a new Version needs to be created called V3 which is copy of version V2 below steps need to be done:

  1. Validate and commit old version V2 as in this example.
  2. Copy the old version and create a new version called V3 from V2
  3. Remove the flag F1 from the old version V2
  4. Map flag F1 to newly created version V3

As the subscription views are created based on the Version Flags and not on Version. So we don’t have to change the subscription views for the newly created version. Downstream application will now be able to fetch data from the new versions instead of old versions.

Posted in Uncategorized | Leave a comment

Reference Data Management Implementation: Four Key Considerations You May Be Overlooking

irawarrenwhiteside:

Great article in discussing Reference Data Management or RDM and it’s relation to MDM. Also important to remember that this is an excellent starting point in organically evolving the underlying processes in support of MDM and a Data Governance organization.

Originally posted on InfoTrellis blog:

In recent years reference data management (RDM) has slowly crept into the forefront of business decision-makers’ consciousnesses, making its way steadily upwards in priority within corporate goals and initiatives. Organizations are suddenly seeing the benefits of investing in RDM, attention grabbed by potential paybacks like smoother interoperability among various functions of the organization and centralized ownership and accountability in creating trustworthy data.

Before we dive into talking about approaches for implementation, I want to look at the potential significance of RDM in an enterprise. When the market is inclined towards concepts like Data Integration, MDM and Business Intelligence tend to hog the spotlight. For these particular corporate initiatives, the primary focus of data is key business information like customers, products or suppliers. It is equally important, however, to appreciate the fact that reference data plays a major role in organizing and comprehending all these key pieces of business data.

Whenever…

View original 846 more words

Posted in Uncategorized | Leave a comment

Informatica Cloud MDM for Salesforce (formerly Data Scout) Review

 

Tactically improving Data Quality and incrementally achieving Data Governance and metadata management  is a natural path and MDM  is the center of that strategy. See Gartner Group’s Applying Data Mart and Data Warehousing Concepts to Metadata Management

 

pyramid

In Metadata Mart the Road to Data Governance or Guerilla Data Governance I outline this approach

I’ve just completed a Data Governance Assessment and review of Informatica CLOUD MDM(formerly Data Scout for Salesforce) with my colleague and excellent Solution Architect Baliji Kkarade .  The client in this case is interested in implementing Informatica CLOUD MDM in Salesforce , as a tactical approach to improving Data Quality and incrementally improving Data Governance .

In general and product is positioned to provide a transactional MDM within Salesforce. We will cover the steps for implementation and some back ground on Fuzzy Matching or de-duplication.

We will walk thru the steps for setting up the tool.

  1. Cloud MDM Settings
  2. Cloud MDM Profile
  3. Adding Cloud related Information to Page Layout
  4. Synchronization Settings
  5. Data Cleansing
  6. Fuzzy Matching and Segments
  7. External Data Sources
  8. Consolidation and Enrichment
  9. Limitations

 

This post assumes familiarity with the Saleforce architecture.

 

1. Cloud MDM Settings

  • Cloud MDM master on/off switch is configured using this setting and other settings like extracting the legal form and domain , overriding Salesforce Account information using Master bean after Match and Merge in Cloud MDM, and Standardizing Country.
  • In some cases, you may wish to turn off Cloud MDM after you have installed and configured it.
  • For example, if you wish to bring in a new set of data without creating beans. To achieve this, you need to switch Cloud MDM off.

mdmsettings

2. Cloud MDM Profile:

  • When Cloud MDM is installed, a default profile is given to all users. In order for your user to get access to all the features of Cloud MDM, you must configure an admin or super user profile. When you implement Cloud MDM, you can use profiles to assign MDM functionality to Salesforce user profiles.
  • Users can have Permissions to Create/Update/Merge/consolidate Account, Contact and Leads, Create/Ignore duplicate Account, Contact and Leads, View consolidated information and create/Edit Hierarchy information

mdmsettings2

3. Add Cloud Related Information to Page Layout:

Helps to add MDM related components like Consolidated view, Find duplicates, MDM related fields like Synchronize, Legal forms, ISO country ,duplicate Account section, Related beans and Master Beans etc .

mdmpageinfos

mdmpageinfos2

4. Synchronization Settings:

  • This setting helps in Synchronizing/Mapping the Salesforce Attributes to Cloud MDM stage Area.
  • We can map Standard fields and 10 custom fields. These standard and custom fields help us in configuring segment settings and match strategy in cloud MDM.
  • Sync job helps creating beans and Master beans in cloud MDM stage Area.

mdmaccountsync

5. Data Cleansing:

Data cleansing ensures the data is in a consistent format. Consistent data improves the quality of reporting and also improves matching results and the accuracy of duplicate detection.

Legal Form :

Legal form normalization is the process of extracting the legal form from the company norm and populating the legal form field with normalized data.

For example, We can configure the legal form field to contain normalized data for business entity designations such as Limited, Ltd., and L.T.D. We can add legal forms to the list available already after profoiling our data set.

Domain Normalization :

We can enable Cloud MDM to populate the domain field with a domain extracted from the website field. Cloud MDM uses the domain field during fuzzy matching.

For example, if a user enters http://www.acme.com/products or www. acme.com in the website field, Cloud MDM can populate the domain field with acme.com. normalized domain ensures domain field consistency and improves match results.

mdmdsatacleansing

6. Fuzzy Matching and Segments:

Segment :

The segment field in the master bean record contains a matching segment. The matching segment is a string of characters that Cloud MDM uses to filter records before it performs fuzzy matching.

To improve fuzzy match performance, Cloud MDM performs an exact match on the matching segments to eliminate records that are unlikely to match. Cloud MDM then performs fuzzy matching on the remaining records. This can be basically creating Categories and Groups in Advanced Fuzzy Matching or “Blocking Indexes” Record Linkage. This will be created for all the Accounts once the Sync between Salesforce and MDM is done. It is also generated for external beans.

Fuzzy matching :

Fuzzy matching can match strings that are not exactly the same but have similar characteristics and similar patterns.

One example of a Fuzzy Matching algorithm is LevenShtein, the original Fuzzy algorithm Levenshtein  Distance or Edit Distinceinvented in 1965

Levenshtein:

Counts the number of incorrect characters, insertions and deletions.

Returns:

(maxLen – mistakes) / maxLen

Levenshtein is a good algorithm for catching keyboarding errors

mdmFuzzy1

Matching is a two step process that determines a match score between two records. First, Cloud MDM performs an exact match on the matching segments to exclude records that are unlikely to have matches. Then, Cloud MDM performs a fuzzy match on the remaining records to calculate a match score between pairs of records. If the match score of the two records achieves the match score threshold, Cloud MDM considers the two records a match.

mdmsegment settingsmdmpageinfos2 mdmpageinfos

7. External Data Source:

We have external data in a system, such as SAP or Oracle EBS. We wish to load this data directly into beans, so we can take some of the information (SIC Code, No of Employees.) from the SAP record, and retain some information (e.g. Company Name) from the Salesforce record.

This setting allows us to configure the external Data source and defining the trust/priority score i.e. which value will win over the other during Consolidation and Enrichment process.

 

7ext

8. Consolidation and Enrichment.

Consolidation :

The consolidated view allows us to look at all beans associated with a master bean. In order to use this view, we must configure the fields that will display in the list of associated beans, as well as the account address information. This is done by configuring field sets.

Enrichment:

This setting allows us to over write the value from the Master bean to the Salesforce Org Account based on the trust/priority score provided during the configuration of the external Data source. We can use the override account option in cloud MDM settings to prevent the automatic override of the Salesforce Org Account accordingly.

8ent

 

9. Limitations.

 

There are two primary limitations:

 

  1. Custom Fields are limited to 10 and only 6 can be used in syncing.
  2. High volume matching from External Source is completed in a “Pre Match” process, which is basically accessing the “Master Bean” externally and developing ETL Process with another tool.

 

Posted in Data Governance, Fuzzy Matching, Guerilla MDM, Guerilla MDS, Mdm, Uncategorized | Tagged , , , , , | Leave a comment

My Wife, My Family , My Stroke (Perception is Perception:Awarement is Reality)

IMG_1099-0 IMG_1098

Six years  earlier the doctors had told my wife(Theresa) she had 18 months to live, she was diagnosed Stage 4 cancer. She is now is full remission.  A women like her mother, she never drank and never smoked.

On June 3, 2014 I had a brain stem stroke(PONS)

I have always told my daughters “Men are idiots”, and apparently I have proven it, here is an example:

5:00 Monday evening my arm start twitching and doesn’t stop.

7:00 Monday arm still twitching, my wife(beautiful, wonderful, courageous  and wise) Tessie, says “Let me take you to the hospital”, of course like any normal “man” I say “Nah it no big deal”

11:00 PM Monday. I take a conference call and notice my left arm and leg are a little numb, obviously I take an aspirin and go back to bed.

5:00 AM Tuesday I plan to catch flight , my wife intervenes and I stay home.

7:00 AM Tuesday I take another conference call, by now I am limping and can’t hold a cup of coffee in my left hand.

8:00 AM Tuesday My wife now says we are going to the  hospital.

8:30 AM Tuesday in hospital I am informed I had a Pons (Brain Stem Stroke) I can now no longer move my left leg or arm and have pretty slurry speech. As a bonus the doctors tell me if I had come in within 5 hours of the first symptom , they may have been able to help, but now it too late.

11:00 PM Thursday I pass out in a diabetic coma in front of my wife and daughter(Victoria), after frantic activity I end up in the ICU, back to square one.

Many illnesses happen in slow motion, this episode of unconsciousness(coma) came very suddenly, if it were not for Tessie’s calmness , control , faith in God and knowledge I am sure I would not be here.

The nurse were unsure what to do , apparently they had given me an extra dose of insulin and very slow to respond.

My wife Theresa(Tessie) with  Victoria’s help literally had to keep me alert and practically drag me to the ICU over the nurses objections.  You know its bad when the nurses are asking you wife for the “Power of Attorney”, which of course she had already given them, and they had lost it.

 

…..25 days later (Diabetics Coma, ICU, Rehabilitation, multiple brain scans , seizure scans, seizures, diarrhea) I am released on the condition I use a walker. Since then I have graduated to a cane,.

I am very lucky I am left with hemiparesis(weakness left side) and double vision(right side)

For 25 days …….. , 25 DAYS …….. my wife(Theresa)  and my entire family never left my side(day of night):

Theresa(Wife, Soul Mate, My everything), Christina(Daughter #1, The Rock). Victoria(Daughter #2, Brainiac ),Julia(Grandaughter #1, Angel sent from heaven),Brandon(Son in Law),Jack (Grandson #1),Jacob(Grandson #2)

Life Summer 2010

 

 

Tessie and our family at St Mary’s in earlier times.

stmarys

 

 

As a man, husband, father and grandfather I have tried to be up to the tasks in providing for my family, sometimes meeting them many times falling short, in my eyes.

My wife led a family effort that as I write about it  and remember it I  am overwhelmed and  tearful. I wish I could say I deserve this , but I can’t I was not good enough.

 

The reality is that we grow older we grow wiser, not through knowledge alone, but through experience.

Apparently Pavlov was “more right” then Plato(Allegory of the Cave). Perception is not reality, Awarement is reality and it come through pain.

This experience has given me the opportunity to eliminate any remnants of ego and permanently instilled a sense of humility. Countless time during my 25 day stay, my family(ALL OF THEM) helped my dress, eat , go to the bathroom(with diarrhea).

As I said one point I went into a coma , low insulin and needed to receive emergency treatment and go to the ICU, it was one  many very traumatic experiences and my wife Tessie never faltered, kept her faith and strength and brought me back.

stroke2photo

Epilogue: At one point I had to prove my cognitive skills were still intact, being that my wife and daughters have a small consulting company Actuality Business Intelligence we collaborated on creating an assessment and power point on how to improve the rehabilitation scheduling system and patient/therapist flow, from my perspective anything that made rehab like work was good.

 

To be clear my hospital room was family “Grand Central Station”

 

Theresa(Tessie),

This post is dedicated to you , I love you with all my heart and soul, forever and ever. God has truly blessed me with you as my soul mate and to receive your love and care.

Your eternally grateful husband  –  Ira

 

two

Ira Warren Whiteside

11/25/2014

 

 

 

 

IMG_1098.JPG

 

 

Posted in Awarement, BrandonStasiewicz, Christina Whiteside, Diabetic Come, family, God, Gospel, insulin, Ira Whiteside, Jack Stasiewicz, Jacob Stasiewicz, Julia Stasiewicz, Pavlov, Plato, PONS Stroke, Soul Mate, stroke, Tessie Whiteside, Theresa Whiteside, Uncategorized, Victoria Stasiewicz | Tagged , | 4 Comments