Ease of Use: The Next Frontier for Event Processing

Featured article by Mark Palmer, VP, General Manager, TIBCO Software, Inc

The future of computing will be event-driven, and recently, we’ve reached a new frontier in its emergence.

In June, the two leading “pure play” event-processing vendors, StreamBase and Apama, were acquired in the same week. These acquisitions prompted industry analysts to ponder who will be the winners in general-purpose analytic platforms for high velocity data. Philip Howard from Bloor Research wondered about the future of StreamBase, about IBM, SAS, and open-source projects like Cassandra, Impala, and Twitter Storm. Each has vastly different capabilities, focus, and ecosystems. Indeed, it’s still very early days for real-time technologies, but there is broad agreement that the next revolution in enterprise technology stacks will be driven by the need for real time.

But before picking a winner, shouldn’t we examine a more basic question: What do Main Street firms need to become more real time?

For the past 10 years, event processing has been growing up on Wall Street. Ten years ago, less than 5% of the world’s trading was algorithmically event-driven; today, over 60% of trading in equities is algorithmic, and much of that is powered by event-processing technologies. But Wall Street IT is famous for its sophistication and resources. Every bank has sophisticated real-time messaging infrastructure, and lots of developers to custom build event-driven applications.

After joining TIBCO, the first thing I did was to talk to their customers. TIBCO literally put real-time messaging on the map 20 years ago, so who better to ask about the future than their customers?

One CIO I met leads IT for a major travel company. After a two-day briefing on event-driven technology, he was excited to take more of his business processes real time. But at the same time, he was concerned: “I see the value in real time, but only if it’s cost-effective to adopt and own. What I really need is a unified, integrated, solution; some kind of real-time-in-a-box.”

Translation: Main Street needs the same real-time capabilities Wall Street needs, but at a fraction of the cost, time-to-market, and project risk.

Dramatically simpler solutions should allow firms to quickly build, test, deploy, and evolve complex real-time systems. Today, most firms with a complete real-time computing fabric have over a dozen products and IT support teams specializing in messaging, stream processing, CEP, event-oriented data storage, caching, integration, end-user analytics, dashboards, mobile access, and alerting.

All those elements need to come together easily. Real-time infrastructure should be deployed on- premise or in the cloud. It needs to scale to handle massive volumes of streaming data and it needs to deliver sub-millisecond responsiveness in order to support machine-to-machine automation. In short, Main Street needs what it took Wall Street 10 years and tens of thousands of developers to create.

Some suggest the answer is open source; my new CIO friend vehemently disagreed: “Open source is the wrong direction for us; I want tools that work straight out-of-the-box. I want frameworks and connectivity that are specific to my problems. Open source gives me a research project to do; I need to get apps running this quarter.”

Products go mainstream when they become accessible. Today, a full real-time stack is too complicated to deploy. Tomorrow, the winners will be the firms that simplify the job of building event-driven applications for Main Street developers.

Mark Palmer

Vice President and General Manager at TIBCO Software Inc.

Mark Palmer is the Vice President and General Manager at TIBCO Software Inc. Previously, Mark was the Chief Executive Officer of StreamBase Systems, which was acquired by TIBCO Software in June 2013. Mark has over twenty years of experience working in the financial technology industry, with expertise in algorithmic trading applications and automated trading architecture development. He has spoken at numerous industry events and is frequently quoted in leading publications, including the Wall Street Journal, Time Magazine, the Financial Times and CNBC. Under his leadership, StreamBase was honored by the World Economic Forum with one of the most prestigious awards in technology innovation – Technology Pioneers. Institutional Investor named Mark as one of “The Top Executives and Innovators in Financial Technology” for four consecutive years, from 2010 to 2013. In 2005 he won an InfoWorld Innovator award and was named to the InfoWorld Media Group’s Innovators Hall of Fame for his achievement in event processing.