Hadoop

Apache Hadoop is a framework for running applications on large cluster built of commodity hardware. The Hadoop framework transparently provides applications both reliability and data motion. Hadoop implements a computational paradigm named Map/Reduce.

Apache Hadoop 2.6.0 enhancements

Hadoop Common
  • Key management server (beta)
  • Credential provider (beta)
Hadoop YARN
  • Support for long running services in YARN.
    • Service Registry for applications.
  • Support for rolling upgrades.
    • Work-preserving restarts of ResourceManager.
    • Container-preserving restart of NodeManager.
  • Support node labels during scheduling.
  • Support for time-based resource reservations in Capacity Scheduler (beta).
  • Global, shared cache for application artifacts.
  • Support running of applications natively in Docker containers (alpha).
Hadoop HDFS
  • Heterogeneous Storage Tiers - Phase 2.
    • Application APIs for heterogeneous storage.
    • SSD storage tier.
    • Memory as a storage tier(beta).
  • Support for Archival Storage.
  • Transparent data at rest encryption(beta).
  • Operating secure DataNode without requiring root access.
  • Hot swap drive: support add/remove data node volumes without restarting data node (beta).
  • AES support for faster wire encryption.