Microsoft does Big Data - Hadoop on Windows

How are the latest trends increasing demand for Data Scientists?

February 9th, 2023

Microsoft does Big Data – Hadoop on Windows

The Hadoop Summit 2012 in San Jose June 13 and 14 could see a the public coming out of Hadoop on Microsoft Windows Server. The move, probably either at Hadoop or at Microsoft’s Tech Ed starting June 11, makes Hadoop, a previously a somewhat exotic big data analytical tool, available on one of the world’s most popular platforms, accessible by Excel, easily linked to SQL Server and its business intelligence, analytical and reporting tools for business intelligence and managed through Active Directory.

Microsoft announced in February that it was partnering with Hortonworks to make Hadoop available on Azure, the Microsoft cloud, and on Windows Servers.

In the announcement, Eric Baldeschwieler, the CTO at Hortonworks, said that expanding Hadoop onto the Windows operating system would broaden adoption of the big data tool.

“We have stated on many occasions our vision that Apache Hadoop will process half of the world’s data within the next five years (or less). The Microsoft contributions are a very important step in making that vision a reality.”

Shaun Connolly, vice president for corporate strategy at Hortonworks, said that Microsoft had to make some hard decision on deciding to go with Hadoop. The company was two and a half years into developing its own proprietary big data platform when it decided to switch to Hadoop, he said.

“Then it was a matter of whom they could partner with to make sure it was optimized properly.” By leveraging the open source community, Microsoft and Hortonworks can make it relatively easy for users to move from open source to Azure, he added.

Financial firms are eager to deploy Hadoop on Windows, especially in real-time systems, he added.

“We have client asking when the Windows version will land because they have existing skills and investments in Windows server.”

Several Fortune 500 companies including major banks have Hadoop working in pilots or production.

Doug Leland, general manager in the server and tools business at Microsoft would not find that surprising.

“A lot of our customers are trying to manage increasing volume, variety and velocity of data. We are also seeing what I describe as the rise of a new set of questions that are being asked of the business, a set of questions that are different in a number of ways,” he said. “One is that they are looking to consume data that is being coming from outside the traditional operational system. Often it is data that lives outside the organization. For example, it is common in packaged good industries to want to know what is being said in social media around their products, their brand.

“We saw Hadoop as as a way to extend SQL 2012 to be able to address this sea of unstructured data,” Leland added. “Microsoft’s strategy is founded on principles of enabling all users to gain insights from any data — structured and unstructured — to surface unique and actionable insights.”