The Business Intelligence space is relatively young, but it is the fastest-growing part of the technology marketplace, and based on data growth is shows no signs of slowing down. Data is doubling every 2 years.
It impacts every part of any business, e.g., CIO, business departments, even security auditors. Like Chief Data Officer (CDO), many new job titles are now part of the HR job description for high-performance companies. These new roles will focus on making data part of the future innovation and are on every CDO’s agenda.
How you can dominate this space will determine your competitive edge in the coming years. Many companies are sitting on mounds of data, secured in massive databases with poor insight and far too few prescribed data-driven actions. If we are going to get to the next level, then we need to understand where we are, and then we can take the next step. Let us proceed to look at the BI Reporting Framework proposed by Stephen Choo Quan, a lead practitioner in the BI space for over 20 years.
There are 5 Levels in the Framework, and each level is gradually more mature as it progresses from Level 1 to Level 5. The maturity will result in effective and efficient consumption of the data by the user. This means the user will arrive at the exact answer faster and with less effort as the BI, Framework matures.
The two units of measurement are TIME and EFFORT. These are extremely important to measure as we want to be actionable to impact the business outcome and not just be analytical and book smart. What is important is that at any level, e.g., Level 4 is a sum of all levels below it in addition to its current level.
The user culture typically defines how far an organization’s maturity gets. Consumer type users tend to max out at level 2, analysts tend to be at levels 3 and 4, and the data scientist tends to be at levels 4 and 5 in the framework. Levels 1-3 tend to focus on data to monitor, control and regulate the business flow. Levels 4-5 creates data architecture as an infrastructure to enable data to become the center for innovation. Let us explore this journey together, and as we do, ask yourself the following:
- Where are we in the maturity level?
- Who are the constituents, their culture, and where do they want to be?
- What can we do today to get to the next level?
- What talent is needed on your team to get to that level, and how do you acquire/leverage it?
- How can we expand from analog revenue thinking to append digital revenue thinking to drive innovation?
- What is the future of data governance for pushing innovation and using data as business assets? (are we owners or stewards to encourage data usage?)
Level 1: GIVE ME THE DATA
The typical characteristic of a Level 1 practice is easy to spot when there are many reports, with about 90% of them not relevant to the user or at best uninteresting to the user.
Many reports are usually explained by the business function trying to cover all possible reporting aspects that it could think of, and the usage of these reports reflects that they are more of clutter than useful.
10% of the reports will be useful to the user, but the user will often have to manually run, wait on render, download a few reports, and then merge the many runs on the desktop. Each takes significant time and effort, each time removing unwanted data then linking data together in context across data sources to arrive at a final desired output.
The main complaint is that users need a custom report from the development team, which can be expensive and time-consuming. Development can be weeks or even months. The data is in the database, but it is hard to get it out and in a useful format to the user/department.
The conditions that lead to this level 1 type build are either a vendor’s ERP that comes with a host of boilerplate reports or a mature department that never cleaned up evolving reports leaving behind a litter of historical reports. A greenfield reporting build with a strong development team and no data architect or an inexperienced architect will lead to level 1.
Level 2: DECLARATIVE REPORTS: OPTIMIZE THE REPORT SPRAWL
Over time the report usage will drive the development team to make relevant reports, new custom reports. Business users can parameterize these reports to reduce the number of similar reports that are merely different by the content but NOT by the structure.
The usage of the reports will be much higher and more valuable. The downloaded content will be in a directly usable format that saves the user hours of work enhancing and filtering data, day over day or week over week.
Users will reduce the clutter and easily find useful, relevant reports. If the user needs changes, they will have to go back to the development team to create another release. Slow processes can be frustrating to the user. The time to get the data is quick until changes are needed.
You can identify Level 2 if you are not able to respond to new questions quickly. A further problem to answer some data trend or point of interest in the data begets data-driven changes.
The conditions that lead to this level 2 type build is an experienced subject matter expert who is vocal and goal-oriented. A business intelligence analytic person augments the development.
Level 3: ADHOC REPORTS: ENGINEER SELF SERVICE THAT ALLOWS FILTERING, SORTING, AND DRILLING
A Level 3 user can select/re-order report columns, create calculated measures, and filter data as desired. The end user’s ability to sort and format reports into the dashboard will dramatically reduce the business’s explanation type analysis.
At this level, these users can become analysts or power users looking for trends in the data as they can ask ad-hoc questions and get relatively immediate answers. Business definitions will start to get standardized as uses securely share reports with other users of similar data access needs.
Scheduling Reports will save time and having multiple delivery methods, including authorized login, email, or FTP. Information will be pulled out of sources on-demand or pushed to the user on named days and times.
You can identify Level 3 by the number of new reports, the number of scheduled reports running, and the amount of data returned as novel ad-hoc questions increase. Users at this level 3 tend to feel empowered and satisfied. The reporting tends to be well-formatted, highly used, and automated. More user capacity leaves room to start thinking of data quality and insights in the data.
The caveat with the report democracy is that there may have variances in calculations, and segmentation criteria are not agreed to and standardized. These can lead to data trust issues if not managed. There is also a strong need to secure master boilerplate reports and the derived data as the reporting insights are companies most valuable assets. Reference data might be categorized differently across different silos and impacts the overall quality of data.
The conditions that lead to this level 3 type build is a strong partnership between mature subject matter experts and business intelligence experts. There is usually the starting of strong demand for data governance and data integration practice at this level. Real-time, high-quality data starts to become the focus as presentation layers become well accepted.
Level 4: DASHBOARDS: STANDARDIZED TERMINOLOGY, DELIVERY ARCHITECTURE AND ESTABLISHED THRESH HOLDS, WHICH MAKE SEAMLESS WORKFLOWS AND ALLOW REAL-TIME MONITORING
Level 4 builds on Level 3 by standardizing many common and useful reports to create common nomenclature and dimension tables to support the business standards. The standardized conformed dimensions allow reports to be interconnected and build relationships and workflow.
If we create hyper-connected data, then more ideas will be generated by the organization. The concept of drilling across is a fundamental level 4 trait. Most tools make drilling up and down hierarchies possible but drilling across but in the context of the dimension requires more maturity. The reports start to share common dimensions as parameters. The complex calculations are localized and made reusable to all reporters.
The most time-consuming part of the visualization delivery cycle is the data preparation and becomes one of the key focus areas at Level 4. This Level is characterized by more advanced data capture, which leads to rich data sets in real-time or pseudo-real-time.
Robust modeling powered by big historical data stores from many different sources is typical traits at level 4. Mature models are also accompanied by rich interactive widgets that can visualize the data points and make them human digestible, making human pre-attentive visualization.
Sophisticated automated Extract Transform and Load (ETL) tools populate mature Data models, which speed up the visualization. In-memory stores are popular in some tools that offer answers at the speed of thought on many device types.
Key Performance Indicators (KPI), KPI composition, and leading indicators are all usually designed and built for real-time monitoring. Digital dashboards are expected at this level, monitoring multiple parts of the business in real-time and highlighting exceptions. Mature enterprises set the thresholds to ensure they align with projected planned goals. Alerts can be pushed to the users or pulled by visiting the dashboard when thresholds are broken.
The conditions for Level 4 are driven by three main factors the need for real-time data (faster data refreshes), quick query times, and the need for an enormous scope of the data domain. The analytics satisfy the explanatory aspects for consumers, and now consumers want to see the data changes faster and data refresh happening as fast as possible. The power users also want to do more data exploration now that many operational KPIs are automated.
Level 5: MACHINE LEARNING & ANALYTIC APPLICATIONS: PROGRAMMATIC PARAMETER DRIVEN STANDARDIZED ANALYTICS THAT DERIVE PRESCRIPTIVE ACTIONS
In the previous 4 Levels, we are a rearview mirror type of analysis focusing on business monitoring, control, and regulation, looking at in the best-case real-time or, in most cases, historical data.
Level 5 seeks to use data for innovation, not just control. It links to level 4 output and flows but adds rules to those workflows to add a prescriptive actionable compliment. How can we predict our future if we do not understand our past?
Level 4 and Level 5 build on each other. Level 5 can interrogate the data to find patterns and anomalies. Machine learning at this Level pushes the term analytics beyond business intelligence. It results in decision trees-based rules on user dimensions (features and labels) to add prescriptive widgets to analytic applications.
The decision trees can be rule-based or machine language defined depending on how static the labels are in the dimension and the complexity of relationships. Google maps application is one such use case where the app can prescribe alternative routes with changing traffic patterns and tell the number of minutes saved in real-time.
This level 5 might engage systems machine learning to train based on historical cases to create predictive decision trees and retain better and more accurate business rules. This information is typically known only by the subject matter experts (SME) with many years of experience. SMEs are the human capital that builds and maintain rules-based trees today, but SMEs cannot scale, but machine learning can scale with little cost overhead.
SME USUALLY DRIVE THE BUSINESS INNOVATION, BUT THE NEXT INNOVATION IDEA WILL MOST LIKELY COME FROM A DATA PATTERN FOUND AT THIS LEVEL 5 PROCESSING.
SMEs TODAY ARE NOW TROUBLED WITH LARGER VOLUMES OF DATA THAN EVER BEFORE, AND THAT VOLUME CONTINUES TO DOUBLE EVERY 2 YEARS. THE LEVEL 5 BI WILL AUGMENT SME KNOWLEDGE WHEN IT FINDS MORE COMPLEX PATTERNS IN THE DATA THAN THE TRADITIONAL AD-HOC QUERIES.
If we have large amounts of data that can support revenue or cost-facing processes, machine learning can help find patterns to optimize the process. Artificial Intelligence will be a critical augmentation to SMEs. Click here for more details.
The Level 5 utopia goal is to get to the segmentation of one. That way, we treat each person as unique and special, but at the same time, we can find categories and influence behavior. Sentiment analysis of the languages will allow structured and unstructured to be mashed together to make richer data sets for the future. I cannot cover this in just one paragraph so that this Level will be continued in this linked article, including big data.
Big data as an asset will possibly drive the future if formatted and packaged for commercial use.
The condition that drives Level 5 is using data as an asset via data exploration. We are yet to derive the actual value of decision support from IT. We have seen it in operations, but we are in the formative years of its effect on decisions.
Summary
Level 1: Many reports, but few are helpful to consumers. Data ritually needs to be augmented or reworked on the desktop. The data is usually operational detailed feeds or downloads.
Level 2: Establish relevant, useful consumer reports but still suffer a long development time, especially for new requests.
Level 3: Consumers become Self-service, and there is a growing number of rich analytics, KPIs, and dashboards that are non-IT driven. Consumer behavior draws insight and new questions.
Level 4: Strong operational monitoring of well-established KPIs that impact positive business outcomes using real-time or near real-time data and analytics. Data governance is well established.
Level 5: Subscription actions based on predictive models of extensive historical data sets. SMEs are augmented with AI in decision-making.
Thanks for reading and sharing ❤
Follow me On: Instagram | Facebook | Linked in
[…] See the Business Intelligence Reporting Framework […]
Comments are closed.