General Challenges

Organizations may arm themselves with information about what they want to implement a solution and even about how to do it, but they may not understand or analyze potential challenges that exist. Since BI search and BI embedded applications have been discussed at length, the focus of challenges will remain mostly in the realm of text mining applications. Obviously a crossover exists in many cases and challenges mentioned here can be applied to other areas within BI as well. Some challenges of implementing text mining solutions include the identification of the appropriate information (to identify what is actually useful), the capturing of context, and the future collection of information.

Because unstructured data is so diverse and expansive it may be difficult to define what data is required. How do organizations identify what potential patterns they are looking for? On the one hand they may be looking for information to help solve a business problem, on the other hand, unknown patterns may be lurking that could help an organization identify potential fraud, ways to increase sales, identify new business opportunities, and develop successful customer experience programs. The issue becomes, how does an organization identify what opportunities to pursue and which ones they are potentially overlooking. How do organizations use text analytics to identify unknown patterns? This means developing a tool to create intelligent text analytics that can identify unknown instances within the organization. Solutions do not provide these answers automatically. Although industry solutions exist, addressing these issues is not intuitive unless an organization defines the specific business rules that are required to access the necessary data.

Even though the proper information is collected, the next challenge involves identifying the context that surrounds the data. Although words or concepts might be identified, making sure that the proper contextual information is collected can be a challenge. Collecting like text strings captures what is requested but may not capture related information. This runs the risk of unrelated results that can lead to wrong assumptions being made.

Text mining initiatives may be shortsighted as organizations look for ways to answer questions to immediate business issues. Consequently, longer term initiatives may require looking at larger data sets or more sources than originally thought. The key challenge for organizations is to identify potential future uses so that expanding text mining applications become a natural extension of the current application.

Conclusion

Organizations looking to implement an unstructured data solution within their BI or BPM environment should evaluate which solution best meets their current and future business requirements. Whether the answer involves BI search, analytics embedded within BI or BPM, or a best of breed text mining or analytics application, management should consider both the short term and long term requirements that align the use of technology with their business issues.

(Copyright 2007 - Dashboard Insight - All rights reserved.)

About the Author

Lyndsay Wise is a senior research analyst for the business intelligence and business performance management space. For more than seven years, she has assisted clients in business systems analysis, software selection and implementation of enterprise applications. She is a monthly columnist for DMReview and writes reviews of leading technologies, products and vendors in business intelligence, data integration, business performance management and customer data integration.