• Votes for this article no votes for this yet
  • Dashboard Insight Newsletter Sign Up

Data Analysis and Unstructured Data
Expanding BI by Thinking Outside of the Box

by Lyndsay Wise, President, WiseAnalyticsMonday, February 14, 2011

Hype surrounding business intelligence and unstructured data ebbs and flows. A few years ago analyzing customer sentiment through text-based analytics was used by a subset of organizations, and independent vendors offering text analytics solutions represented a niche industry that was external to mainstream BI. Organizations adopting text analytics tend to apply solutions externally to BI or structured data analytics. Generally speaking, their goals are to identify customer sentiments, potential risk, and overall trends identification. In some cases companies do integrate various types of analytics to get a broader picture of what is happening within the organization.

In reality though, for BI to be effective both structured and unstructured data are required. Because so much information resides in emails and other documents, the only way to get a full view of what is happening within a specific business unit or the organization as a whole is by combining financial and other performance related data with documents, geospatial data, and other types of information. Consequently more BI solutions integrate unstructured options into the available types of data capture. This leads to greater diversity in the types of analytics that can be conducted and in turn helps increase the variety of analytics.

This article examines how organizations are starting to integrate unstructured data within their BI infrastructure. This includes identifying some applications of text analytics and other forms of unstructured information, as well as what businesses can look forward to in the future of BI and the broader integration of unstructured related analytics. 

BI and unstructured data overlap

With the addition of social network analysis and the increase in geospatial and location intelligence, to stay on top of the competition businesses require the ability to integrate these aspects of analytics into their current BI infrastructure. Unfortunately many companies do not have mature BI environments conducive to adding unstructured data sources. With organizations still struggling to get valuable information out of business intelligence that aligns itself with overall metrics and performance management, the addition of unstructured data might be too much to handle. 

Before organizations expand into unstructured data analysis they need to make sure that their current BI infrastructure is actually working for them. Once businesses decide to take the plunge, many considerations are required. The general considerations to look at when adding unstructured analysis within a structured environment are broad. These include customization, integration, development, etc. In essence these are the same requirements for developing a traditional BI or general analytics solution.

General applications of unstructured data

Currently the type of analytics that intuitively integrate unstructured data analysis is geospatial mapping. Due to the proliferation of dashboards and visualization, Google Maps and graphics representing geographic regions are becoming quite common. Their use can be quite diverse and include location intelligence in terms of identifying potential store placements, looking at buying trends by zip code, etc. In addition metrics can be broken down by geographic region to identify sales performance, government spending, or general statistics.

Beyond GIS and general mapping, identifying customer sentiments from notes sections are becoming more important as businesses look outside of the organization to identify what is being said about products and services within social networking channels. In addition, the combination of external and internal customer sentiment sources create a more balanced view about what is happening in relation to overall customer experience. Industries requiring heightened sensitivity to potential fraud from both internal and external sources can identify trends in submitted documents, email messages, and general content management systems. No matter what the application, the reality is that as BI use in general matures, so does the way in which businesses look at all areas of analytics and the types of data that can be integrated within the BI infrastructure.

What should organizations look for?

Adoption of unstructured data within a BI framework is small at best. Businesses still struggle with how to get the most out of analytics, and adding more information that is complicated can lead to a higher probability of project failure. In addition, many organizations forget about context when they are looking at information. The best example is related to dashboards and the fact that many companies identify the metrics they are looking for but are unable to quantify the benefits of monitoring those specific metrics (and how they fit within the broader picture). The same issue exists for companies that want to tackle unstructured data analysis. The first step is to identify the context e.g. looking at the questions that need to be answered.

Once context is defined businesses should consider the traditional technical requirements as well as interactivity.  Questions such as some of the following should be asked to identify proper starting points:

  • Who will be interacting with the solution, and what is their level of expertise with analytics?
  • What solutions already exist in-house? This helps identify whether a current application has additional functionality to take into account unstructured content. It also helps to identify what IT infrastructure exists so that the proper complementary solutions can be considered within the software evaluation.
  • What types of information will be integrated?
  • What is the reason behind integrating unstructured data within a BI framework?
  • What business pain is being faced? This also includes the goal of using unstructured data. In addition, by looking at the types of information and the whys behind it, companies can see whether simple solutions exist off the shelf or whether a lot of customization will be required.

Where the market seems to be headed

Due to constant changes in technology and the increasing focus on social network analysis, organizations will start to take unstructured and external data sources more seriously when considering their overall analytics platform. Because unstructured data represents such a large portion of data stored within the firewall, the only way to get value out of information is to include both structured and unstructured data sources to ensure a broader perspective. Consequently this means that once companies mature in their overall analytics use, adding unstructured data sources becomes the next step towards broader business visibility.

About the Author

Lyndsay Wise is an industry analyst for business intelligence. For over seven years, she has assisted clients in business systems analysis, software selection and implementation of enterprise applications. Lyndsay is the channel expert for BI for the Mid-Market at B-eye-Network and conducts research of leading technologies, products and vendors in business intelligence, marketing performance management, master data management, and unstructured data. She can be reached at lwise@wiseanalytics.com. And please visit Lyndsay's blog at myblog.wiseanalytics.com.


(Copyright 2011 - Dashboard Insight - All rights reserved.)

Tweet article    Stumble article    Digg article    Buzz article    Delicious bookmark      Dashboard Insight RSS Feed
 
Other articles by this author

Discussion:

No comments have been posted yet.

Site Map | Contribute | Privacy Policy | Contact Us | Dashboard Insight © 2014