Introducing Hadoop and Big Data into a Traditional Data Organization: A True Story and Learned Lessons

In this talk we will share our story about our journey into Hadoop, Big Data, Map Reduce, and NoSQL with the goal to reduce cost and improve speed of data processing. In our journey, we start with Hadoop and its Map Reduce algorithm that splits processing across many commodity machines. We found this to be an effective solution, albeit not without warts. From our experiences, you will learn how to effectively introduce Hadoop into a conventional data processing organization and about the approaches to integrate it not only with conventional data processing technologies, but also with people. As our appetite grew, we had to reach towards NoSQL databases, and there we discovered Apache Cassandra – a distributed high performance data store that we use together with Hadoop. You will learn about how to migrate from local deployment to the cloud. We conclude with "7 Habits of Successful Hadoop Projects".

Hadoop and Big Data: why do we care?
How is Hadoop and Map Reduce different?
Avoiding friction and integrating with relational databases
Preventing shock: the people issue
The buzz of Hive
When Hadoop alone is not enough: adding NoSQL / a case for Cassandra
Next: into the Cloud!
7 Habits of Successful Hadoop Projects

Dr. Vladimir Bacvanski has two decades of engineering experience with software and data technologies in areas such as architecture and design of mission critical and distributed enterprise systems, rule-based systems and languages, modeling tools, real-time systems, agent systems, and database technologies. Vladimir has helped a number of companies including US Treasury, Federal Reserve Bank, US Navy, IBM, Dell, Hewlett Packard, JP Morgan Chase, Nokia, Lucent, Nortel Networks, General Electric, BAE Systems, AMD, and others to select, transition, and apply new software technologies. Vladimir is published worldwide and is a frequent speaker, session chair, and workshop organizer at leading industry events.

Jeff McCalip is an Enterprise Architect with VHA Inc., with over 25 years in various roles around data architecture and management. Jeff is involved in all aspects of data architecture, including business rules, ontologies, data integration, data quality, and business intelligence. He has led teams in the implementation of data warehousing and business intelligence applications with recent emphasis on large volume distributed data processing.

Lloyd Mangnall has been architecting and implementing complex technology solutions across multiple industries for more than 20 years and is currently VP of Enterprise Architecture for VHA, Inc. He has been a leading advocate for the use of Service Oriented & Model-Driven approaches to software architecture & implementation for the past 13 years. His recent activities involve Big Data projects with Hadoop combined with semantic technologies.