X
Tech

Intel baking Apache Hadoop into silicon for big data, security uses

Silicon Valley is turning to Hadoop to tackle big data, and Intel is attempting to get involved at ground level through integration on its chips.
Written by Rachel King, Contributor

SAN FRANCISCO -- Not to be left out of the party this week, Intel revealed an Apache Hadoop distribution of its own during an invite-only event on Tuesday morning.

See also: Hortonworks delivers beta of Hadoop big-data platform for Windows | EMC launches Hadoop distribution, takes aim at Cloudera | HP joins Hadoop party with security plug-in for ArcSight

"We're in an era of generating huge amounts of data," said Boyd Davis, vice president of Intel's Architecture Group. "But the key is not what we get out of that."

"We're in an era of generating huge amounts of data," said Boyd Davis, vice president of Intel's Architecture Group. "But the key is not what we get out of that."

Citing that the point is really big answers, Boyd acknowledged that you could look at big data as just another big buzzword.

But he followed up that big data also has the potential to transform business models and society overall, from personalized healthcare to managing scarce energy supplies.

Like EMC and Hewlett-Packard earlier this week, the overarching idea behind Intel's distribution is to massive amounts of big data for the purpose of enabling better business decisions while also identifying potential security threats more quickly.

Furthermore, the big picture for Intel is to beef up its portfolio for the datacenter -- both for analytics purposes as well as offering a framework that can connect and manage devices within a entire corporation in a scalable manner.

But Intel is framing its deployment of the open source software framework as a ground-up approach by baking Hadoop directly into the silicon level.

The Santa Clara, Calif.-based corporation explained that it is utilizing Hadoop because it is open and scalable, thus making it a plum strategy for handling evolving datacenter challenges in the enterprise space.

In many cases now from Hadoop to OpenStack, Davis described we're seeing the open source technology driving leads in figuring out scale for high-performance computing and the cloud.

Arguing that the Hadoop framework, in particular, has enormous potential, Davis said that Hadoop will be a foundational layer within organizations that they can build a variety of stacks on top of through a horizontal distribution.

Intel added that deploying and managing this Intel-Hadoop distribution should be simple for IT managers because it is "automatically configured to take the guesswork out of performance tuning."

The Ultrabook maker described that it optimized its Xeon chips, in particular, for networking and I/O use cases to "enable new levels" of data analytics.

For example, Intel estimated that one terabyte of data could be analyzed within seven minutes through this Hadoop distribution compared to four hours previously.

Intel estimated that one terabyte of data could be analyzed within seven minutes through this Hadoop distribution compared to four hours previously.

Aside from promising encryption up to 20 times the speed with AES-NI, Davis outlined that Intel's Hadoop distribution framework will be optimized with solid state drive and cache acceleration, up to 8.5 times faster queries in Hive, hardware-enhanced compression with AVX and SSE 4.2, and automated tuning on clusters with Active Tuner from Intel Labs.

On the security side, the microprocessor giant is boasting that its distribution is the first to offer "complete encryption" on its Xeon processors, promising enterprise customers that their data sets will be secure without sacrificing performance power.

Also like EMC, Intel rolled off a list of nearly two dozen partners at announcement time to deliver this deployment to public and private cloud environments. Some of those partners include Cisco, Dell, and SAP.

Davis admitted that "there are other people far more capable than we are" to work at the applications solutions level, but he asserted that Intel will take care of the foundational level.

In order to speed up interest in and subscriptions for its deployment, Intel revealed that it is also investing in smaller big data companies, such as MongoDB and Guavus Anaytics, to build new analytics solutions based on Apache Hadoop software.

Set to roll out worldwide, the Intel-Hadoop deployment will be delivered through an annual subscription with technical support through solution vendors and service providers.

Editorial standards