Inside the Briefcase

How Security in Tech is Being Reinforced

How Security in Tech is Being Reinforced

In an increasingly digital world, security has become a...

2022 Business Spend Management Benchmark Report

2022 Business Spend Management Benchmark Report

Read the 2022 Coupa Benchmark Report to explore 20...

Cloud Security: Understanding “Shared Responsibility” … and Keeping Up Best Security Practices

Cloud Security: Understanding “Shared Responsibility” … and Keeping Up Best Security Practices

Cloud computing has been around for many years now,...

Webcast: HOW TO SCALE A DATA LITERACY PROGRAM AT YOUR ORGANIZATION

Webcast: HOW TO SCALE A DATA LITERACY PROGRAM AT YOUR ORGANIZATION

Join data & analytics leaders from Starbucks, Cardinal Health,...

How EverQuote Democratized Data Through Self-Service Analytics

How EverQuote Democratized Data Through Self-Service Analytics

During our recent webinar on scaling self-service analytics, AtScale...

How Cloud Computing Democratizes Big Data

June 18, 2013 No Comments

Financial firms have been crunching Big Data for years – with the help of cloud computing, now it’s everyone else’s turn.

Big Data, just like Cloud Computing, has become a popular phrase to describe technology and practices that have been in use for many years. Ever-increasing storage capacity and falling storage costs – along with vast improvements in data analysis, however, have made Big Data available to a variety of new firms and industries.

Scientific researchers, financial analysts and pharmaceutical firms have long used incredibly large datasets to answer incredibly complex questions. Large datasets, especially when analyzed in tandem with other information, can reveal patterns and relationships that would otherwise remain hidden.

Extracting Simplicity From The Complex

As a product manager within the Global Market Data group at NYSE Technologies, I was consistently impressed with the how customers and partners analyzed the vast sets of market trade, quote and order-book data produced each day.

On the sell side, clients analyzed data spanning many years in an attempt to find patterns and relationships that could help fund portfolio managers build long-term investment strategies. On the buy side, clients mined more-recent data regarding the trade/quote activities of disparate assets. University and college clients sought data spanning decades. Regardless of the specific use case, clients required technology to process and analyze substantial and unwieldy amounts of data.

Various technologies are employed to meet the needs of these various use cases. For historical analysis, high-powered data warehouses such as those offered by 1010data, ParAccel, EMC and others, are incredible tools. Unlike databases, which are designed for simple storage and retrieval, data warehouses are optimized for analysis. Complex event processors such as those from One Market Data, KDB and Sybase give high-frequency and other algorithmic traders the ability to analyze market activity across a wide array of financial instruments and markets at any given microsecond throughout the trading day.

These technologies are now being deployed within new industries. Business intelligence tools such as those offered by Tableau and Microstrategy can now deal with very large and complex datasets. To a lesser extent, even Microsoft Excel has been retooled to handle Big Data with newly architected pivot tables and support for billions of rows of data within a single spreadsheet.

But Big Data is useful only if analysts ask the right questions and have at least a general idea of the relationships and patterns Big Data analysis may illuminate.

(See also Blinded By Big Data: It’s The Models, Stupid.)

Do You Need Big Data?

Is Big Data right for your company? The first question any firm must ask is if they will benefit from Big Data analysis. Begin by understanding the data sets available to you. Analysis of 20 years of stock closing prices, for example, would not likely require the power of Big Data systems. Given the relatively small size of this dataset, analysis can, and probably should, be performed using SQL or even simply Excel.

Read Full Article

 

Blogs, DATA and ANALYTICS 

Leave a Reply

(required)

(required)


ADVERTISEMENT

Gartner