IT Briefcase Exclusive Interview: Managing Event Big Data with Charles Rich, NastelSeptember 12, 2012 No Comments
Charles Rich, Vice President of Product Management, Nastel
Big Data is one of the “hottest” topics in the IT space these days, but along with all of the wonderful benefits it offers, come challenges as well.
In the below interview, Charles Rich from Nastel talks about Big Data today, and how to overcome some of these challenges.
- Q: Although “Big Data” has been around for a very long time, the concept of managing “Event Big Data” is reasonably new. Can you please explain the difference between these two concepts, and the importance of analyzing the “data about the data”?
A: Big Data itself is not new. Retailers have struggled for years to analyze volumes of buyer demographics, behavior, purchasing history, product inventories, assortments and logistics in order to optimize the cost of sales. Over time this has only become worse with almost everything available on the internet.
However, a not so obvious problem is the one of event big data. Business applications automating tasks such as online purchasing, trading, accounting and so many other vital processes are monitored by application performance monitoring (APM) systems. When these applications have problems, the APM systems generate events in order to alert IT operational staff. This event data is usually saved in logs and databases. As more applications are monitored, the complexity increases, more failures occur and more events are generated about them. Often, enterprises have overlapping monitoring systems that produce different events about the same failures with even different interpretations. Eventually, an enterprise gets to the point where it is receiving more events each day about its applications than it can possibly process. Thus, an enterprise’s real-time awareness of the state of its business erodes. The enterprise no longer knows exactly what is going on.
Many large enterprises, especially in financial services, have begun to be effected by this.
- Q: Why, in your opinion, is it so important for companies that deal with high-volume transactions on a daily basis to have an Application Performance Monitoring solution?
A: It is essential for firms whose applications are generating a large volume of transactions to stay on top of their status. The only way to do this is with an application performance monitoring solution. Monitoring the network or just resource utilization won’t be sufficient. Many of these enterprises generate millions of transactions each day. They require an automated way to instantly detect a trend toward a problem condition and analyze why this problem occurred so it can be repaired immediately. In many industries there are regulatory issues that specify very short time windows for problem resolution. For example, the Dodd-Frank Act requires brokerage firms to report bond trades to NASD within 15 minutes. That is a very small window to detect something went wrong, correct it and then re-report it.
- Q: What verticals do you think benefit the most from this type of solution?
A: Industries such as financial services, healthcare and eCommerce can all benefit from this. These industries share a common underpinning, the “order process.” The APM solution is used to monitor claims as they are processed, online orders and, of course, trades.
- Q: How can Nastel’s product family, AutoPilot, help companies detect trends that enable problems to be discovered before users are impacted?
A: Nastel baked real-time analytics into its AutoPilot software from the very beginning. This is a low-latency, pattern matching analytics based on Complex Event Processing. It can easily handle millions of events per second and scan them for patterns that illustrate a potential problem. Its trend analysis is constantly updated, always reflecting what is “business normal” and what is not. And it does that without slowing down the user’s applications.
Charles Rich, vice president of product management and marketing at Nastel, is a software product management professional who brings over 27 years of technical hands-on experience working with large-scale customers to meet their application and systems management requirements.DATA and ANALYTICS , Fresh Ink