Technology advancements allow business intelligence to be applied more broadly and to include types of analysis never thought possible when it was first developed. This is especially true of unstructured data. In the past, structured data sources were valuable enough on their own to provide organizations what they required to understand their company’s performance and to plan for the future. Now, using structured data alone might not be enough to provide decision makers with a complete view of their company’s landscape.
The defined role of text analytics within BI has been somewhat elusive. At various periods in recent years, the convergence of text and numerical data was seen as an important aspect to providing enhanced BI value. However, the adoption rate of this concept remains limited at best. Consequently, many organizations do not use unstructured data within their business intelligence framework, meaning that the information they analyze is limited to sales-oriented or transaction-based data. Although this provides general insights, an overall view of the customer, product or services offered cannot be seen without an inclusion of unstructured data sources.
This article looks at the convergence of text analytics within the framework of business intelligence. This includes looking at practical applications of text analysis and the added benefits it offers organizations when coupled with BI and structured analytics.
Realistic now or future trend?
The idea of being able to take data from multiple sources, combine it and provide analytics to give businesses a competitive edge is the dream of many BI enthusiasts and decision makers alike. Organizations want easy fixes and ways to integrate multiple data sources and to combine that data in a visibly pleasing way to provide visibility into sales, operations, customer relationship management, etc. When looking at structured data sources, most BI providers support multiple data sources, such as Oracle, SAP, Microsoft, Teradata, IBM, and Sybase. Companies now expect that they will be able to access and combine data from all of those sources to gain an understanding of what is happening within the organization.
These same expectations do not exist when considering the use of unstructured and text-based data. Even though some BI solution providers may support certain types of unstructured information, the reality is that unstructured data needs to be structured before it can be used as part of any analysis. More effort is required to combine unstructured and structured sources. Activities such as integration, development and customization may take longer when looking at combining these various data sources. Extra time to deploy solutions may be seen by organizations as a drawback to implementing BI that includes unstructured information.
Unfortunately, the extra effort required to integrate unstructured data into the overall BI environment means that many organizations choose to stick with what they know works. Even though organizations can integrate text analysis into their overall analytics use, widespread adoption is still many years off. This does not mean that some companies are not using text analytics. Customer sentiment analysis and general call centers or help desks are examples where it becomes important to take notes within comments fields and develop trends based on customer behavior, satisfaction and long-term customer value. For organizations with a focus on retail, healthcare or fraud detection, the ability to identify text along with numeric data becomes important to provide better service or to detect suspicious behavior.
Today’s advanced applications of text analytics within business intelligence are limited. For the few companies that apply text analytics within their BI infrastructure, most use it to determine customer sentiment analysis or to detect fraud. Although there are companies that use text analytics in other ways, adoption of text within BI rarely extends beyond these applications.
By looking at comments fields within call center logs, organizations can identify trends based on service levels, satisfaction in relation to specific products, quality control issues, etc. Looking at text records helps organizations remain proactive by identifying trends to stave off potential problems - or alternatively, to take action on successful projects or initiatives.
Early fraud detection saves companies and people billions of dollars a year – whether looking at insurance, healthcare, etc. Organizations need ways to identify trends based on submitted forms and customer activity. The ability to take text and predict suspicious behavior enables businesses to develop proactive measures surrounding fraud instead of having their fraud initiatives focused on rectifying the situation after the fact.
Getting the most out of text
Many organizations start to consider text when they want to take their overall analytics to the next level. The combination of structured and unstructured data enables companies to combine both types of information to help drive better customer and competitive-focused analyses. Organizations can learn what people are saying online about their products, their competition and the market in general. By combining this information with customer sentiment analysis, product design and customer service ratings, both product quality and customer perception can be increased. Although not necessarily easy or quick to deliver, the development of initiatives that combine various data sources and types of data to drive business proactively is worth the effort.
Considering text and numerical data analysis
For organizations considering text analytics, some questions should be asked as a starting point:
- Why is the organization looking at text-based information?
- What business problem needs to be solved?
- What information is required? And where does it come from?
- What type of information is desired – unstructured as well as structured? This might include looking at customer product ratings and combining that with overall product sales and penetration.
- What processes are required to gather and to validate this data?
- What types of metrics are being evaluated? Or alternatively, what types of analytics are required?
These questions do not address all issues that need to be identified, but do provide a starting point for organizations looking to add text analytics as part of their overall BI use. Although not right for all businesses, the addition of text analytics can help many organizations gain broader insights into their overall operations.
About the Author
Lyndsay Wise is an industry analyst for business intelligence. For over seven years, she has assisted clients in business systems analysis, software selection and implementation of enterprise applications. Lyndsay is the channel expert for BI for the Mid-Market at B-eye-Network and conducts research of leading technologies, products and vendors in business intelligence, marketing performance management, master data management, and unstructured data. She can be reached at firstname.lastname@example.org. And please visit Lyndsay's blog at myblog.wiseanalytics.com.
(Copyright 2009 - Dashboard Insight - All rights reserved.)