Inside the Briefcase

Augmented Reality Analytics: Transforming Data Visualization

Augmented Reality Analytics: Transforming Data Visualization

Tweet Augmented reality is transforming how data is visualized...

ITBriefcase.net Membership!

ITBriefcase.net Membership!

Tweet Register as an ITBriefcase.net member to unlock exclusive...

Women in Tech Boston

Women in Tech Boston

Hear from an industry analyst and a Fortinet customer...

IT Briefcase Interview: Simplicity, Security, and Scale – The Future for MSPs

IT Briefcase Interview: Simplicity, Security, and Scale – The Future for MSPs

In this interview, JumpCloud’s Antoine Jebara, co-founder and GM...

Tips And Tricks On Getting The Most Out of VPN Services

Tips And Tricks On Getting The Most Out of VPN Services

In the wake of restrictions in access to certain...

How Cloud Computing Democratizes Big Data

June 18, 2013 No Comments

Financial firms have been crunching Big Data for years – with the help of cloud computing, now it’s everyone else’s turn.

Big Data, just like Cloud Computing, has become a popular phrase to describe technology and practices that have been in use for many years. Ever-increasing storage capacity and falling storage costs – along with vast improvements in data analysis, however, have made Big Data available to a variety of new firms and industries.

Scientific researchers, financial analysts and pharmaceutical firms have long used incredibly large datasets to answer incredibly complex questions. Large datasets, especially when analyzed in tandem with other information, can reveal patterns and relationships that would otherwise remain hidden.

Extracting Simplicity From The Complex

As a product manager within the Global Market Data group at NYSE Technologies, I was consistently impressed with the how customers and partners analyzed the vast sets of market trade, quote and order-book data produced each day.

On the sell side, clients analyzed data spanning many years in an attempt to find patterns and relationships that could help fund portfolio managers build long-term investment strategies. On the buy side, clients mined more-recent data regarding the trade/quote activities of disparate assets. University and college clients sought data spanning decades. Regardless of the specific use case, clients required technology to process and analyze substantial and unwieldy amounts of data.

Various technologies are employed to meet the needs of these various use cases. For historical analysis, high-powered data warehouses such as those offered by 1010data, ParAccel, EMC and others, are incredible tools. Unlike databases, which are designed for simple storage and retrieval, data warehouses are optimized for analysis. Complex event processors such as those from One Market Data, KDB and Sybase give high-frequency and other algorithmic traders the ability to analyze market activity across a wide array of financial instruments and markets at any given microsecond throughout the trading day.

These technologies are now being deployed within new industries. Business intelligence tools such as those offered by Tableau and Microstrategy can now deal with very large and complex datasets. To a lesser extent, even Microsoft Excel has been retooled to handle Big Data with newly architected pivot tables and support for billions of rows of data within a single spreadsheet.

But Big Data is useful only if analysts ask the right questions and have at least a general idea of the relationships and patterns Big Data analysis may illuminate.

(See also Blinded By Big Data: It’s The Models, Stupid.)

Do You Need Big Data?

Is Big Data right for your company? The first question any firm must ask is if they will benefit from Big Data analysis. Begin by understanding the data sets available to you. Analysis of 20 years of stock closing prices, for example, would not likely require the power of Big Data systems. Given the relatively small size of this dataset, analysis can, and probably should, be performed using SQL or even simply Excel.

Read Full Article

 

Leave a Reply

(required)

(required)


ADVERTISEMENT

Gartner

WomeninTech