I have recently had the opportunity to help in clarifying the issues and techniques in adapting an Agile/SCRUM “in progress” ENTERPRISE Data Warehouse project via the Extreme Scoping methodology from Larissa Moss. Thankfully I have been working with an excellent Scrum Master Andrew Ruffalo.

Please see the articles below one does an excellent job in providing an explanation of how you can modifyAgile, using a well defined and proven approach of Extreme Scoping as well as an article on Information Architecture, the last one is a presentation I gave with Aaron Zornes on the use of Metadata Marts.

Also here is a brief treatment of the these areas as I see it:

First we will have several layers of Data/Information Models spanning the Enterprise from left to right. they are as follows:

Staging – SourceOriented

Enterprise – Target or Business Oriented in our case this this will probably be based on some type of Common “Canonical” Model (ie Accord?)

Data Mart – A “Pure Dimensional” model consisting of Conformed Dimensions and Purposed Facts for your Business Model and mapped back to the EDW And the “Canonical” model.

Analytical – This will also map back to the “canonical” model and will be physically represented in an OLAP cube.

MetaData Mart spanning all models and supporting complete linage and mappings between all layers, including the “canonical model”

If we agree on these layers then it is relatively straight forward to follow an Agile\Extreme approach , creating User Stories (Business Questions) within Epics consisting of a few Metrics , Dimensions and Hierarchies and in essence implementing them in the Staging and EDW, Data Mart and Cubes.

The technique is to fully population the Staging Model, “sparsely populate” the EDW(Common Model) , which will be abstracted from a “canonical” model,incrementally design and load the Data Mart(Data Mart, Cube). there will be consist “refactoring” of the entire Application.

The overwhelming priority has to be supporting the “Information Value Chain”, and the Business Capabilities Matrix as the driver.

Information Value Chain: Objectives => Business Question => Metric => Dimension => Characteristic

Business Capabilities Matrix: Epics,Stories across the top Metrics and Dimensions down the side, leading to intersection of Business Capabilities and Dimension Model

.
I believe we need to stay true to these principles, obviously you need to incorporate your teams experiences and knowledge in aligning there approaches and plans with the approved Solution Architecture and Information Architecture as well as following proven and verifiable “best practice” several are listed below..

This will enable advanced predictive analytics, financial modeling as well as data mining by incorporating data lineage as well as data quality, delivered and verified by the Business Product Owner and Sponsors very quickly.

In the coming weeks we will be detailing the specifics techniques we used and diving into what worked and what didn’t.

Articles / Webinars

Beware of Scrum Fanatics On DW/BI Projects – Larissa Moss

Beware of Scrum Fanatics On DW/BI Projects

Information Architecture: Do Not Model the Staplers – David Loffredo
http://msdn.microsoft.com/en-us/library/cc168600.aspx

Creating a Metadata Mart the Road to Data Governance. -Aaron Zorness / Ira Warren Whiteside
http://blog.melissadata.com/data-quality-authority/2010/11/free-webinar-creating-a-metadata-repository-metadata-mart-on-the-road-to-data-governance.html

Creating a Metadata Mart incorporating Data Governance , Data Profiling , Code Generation via SQL, SSIS – Ira Warren Whiteside
http://www.sqlpass.org/Events/ctl/ViewEvent/mid/521.aspx?ID=656