Rda case statement proposal for



Download 29.29 Kb.
Date conversion29.04.2016
Size29.29 Kb.
D R A F T

RDA CASE STATEMENT PROPOSAL FOR

Big Data Infrastructure Working Group (BDI-WG)

Wo Chang, NIST, December 10, 2013


Co-Chairs: Wo Chang (NIST), Geoffrey Fox (Indiana U.), Yuri Demchenko (U. of Amsterdam)

Background


Big Data is the term used to describe the deluge of data in our networked, digitized, sensor-laden, information-driven world. There is a broad agreement among commercial, academic, and government leaders about the remarkable potential of “Big Data” to spark innovation, fuel commerce, and drive progress. The availability of vast data resources carries the potential to answer questions previously unobtainable. However there is also broad agreement on the ability of Big Data to overwhelm traditional approaches. The rate at which data volumes, speeds, and complexity are growing is outpacing scientific and technological advances in data analytics, management, transport, and more.
Despite the widespread agreement on the opportunities and current limitations of Big Data, the lack of a best practice implementation guide will hold back future Big Data application deployment advancements. How to curate or pre-process data at rest or in motion from a central location or distributed sites? How to decide whether to transfer large datasets or analytic tools between data storage and process site? How best to provision and configure computing cluster and resources? What security and privacy measures are needed? How to manage and monitor massive computing nodes from the traditional computing environment?
  1. WG Charter


Big Data architectures come in many shapes and forms ranging from academic research settings to product-oriented workflows [1]. An interoperable Big Data architecture, such as the NIST Big Data Reference Architecture (NBD-RA) [2], and other similar standard frameworks are desirable because they could provide vendor-neutral, technology and infrastructure agnostic environment to enable Big Data stakeholders to pick-and-choose best analytics tools for their processing under the most suitable computing platforms and clusters while allowing value-additions from Big Data service providers and flow of data between the stakeholders in a cohesive and secure manner.
The focus of the BDI-WG is to establish best practices implementation guidelines for how to deploy and manage big data applications using NBD-RA and other big data architectures along with best technologies available today to meet the ever challenging big data application demands. The rate at which data volumes, speeds, and complexity are growing requires new forms of computing infrastructure to enable breakthrough discoveries and innovation in ways that enhance economic security and improve quality of life.
  1. Value Proposition


Big Data best practice implementation guide will provide data scientists the best guideline on how to orchestrate Big Data applications across a diversified range of domains using a systematic approach with a generic reference architecture. This will provide information on how best to integrate various scalable Big Data technologies as well as automatic the provisioning and configuring of computing, resources, infrastructure, and workflow between Big Data architecture components.
There are a wide range of actors with specific roles and responsibilities/activities involved to design, develop, deploy, and manage big data applications.
NIST Big Data Reference Architecture is comprised of five key components including: (a) System Orchestrator, (b) Data Provider, (c) Big Data Application Provider, (d) Big Data Framework Provider, and (e) Data Consumer. Furthermore, Security and Privacy applies to all components. There are many actors within each of the component including:
System Orchestrator

  • Business Leadership

  • Consultants

  • Data Scientists

  • Information Architects

  • Software Architects

  • Security Architects

  • Privacy Architects

  • Network Architects

Data Provider

  • Enterprises

  • Public Agencies

  • Researchers & Scientists

  • Web, FTP, etc. Applications

  • Network Operators

  • End Users

Big Data Application Provider

  • Application Specialists

  • Platform Specialists

  • Consultants

Big Data Framework Provider

  • In-house Clusters

  • Data Centers

  • Cloud Providers

Big Data Security and Privacy

Data Consumer

  • End Users

  • Researchers

  • Applications

  • Systems
  1. Engagement with existing work in the area


Engaging and collaborating with other related existing works are important to our mission; they include consortium, alliances, and standards bodies; they can excel our development with high quality best practices. These include:

  • NIST Big Data Reference Architecture [2]

  • Apache Big Data Stack (Big Top) [3]

  • Berkeley Data Analytics Stack [4]

  • Ubuntu JuJu Framework [5]

  • Dell Crowbar Framework [6]

  • Puppet Labs [7]

  • Etc.
  1. Work Plan





  • Milestones

  1. M3: Use Cases Gathering

  2. M3: Technologies Gathering

  3. M6: Testbed Implementation (Mapping solutions – combination of technologies – to NIST Reference Architecture)

  4. M9: Operation Monitoring and Management

  5. M12: Establish initial Big Data Best Practice Implementation Guide

  6. M15: Revise Big Data Best Practice Implementation Guide

  7. M18: Final version of Big Data Best Practice Implementation Guide




  • Mode and Frequency of Operation

It is anticipated that all discussions will be conducted via emails plus 2-hours bi-weekly teleconferencing through a Web-based conferencing tool. The date/time of the bi-weekly telecon will be determined by Doodle Poll.


  • Membership

Participation in the WG is open to all interested parties. There are no membership fees.


  • Coordination/Interaction

The WG will function in close coordination with other Big Data related standards and best practices from industry, academia, and government at the international level.


  • Standing Rules

All information exchanged within the WG will be non-proprietary.

All information exchanged within the WG will contain non-PII materials.

WG members should assume that all materials exchanged will be made public.

Documents will be publicly accessible from the WG portal.


  1. Adoption Plan


Strong collaborations are in place with the NIST Big Data Public Working Group (NBD-PWG) who initially created the NBD Reference Architecture (RA), and is currently seeking partnerships in order to validate the architecture model. BDA-WG will be a perfect platform to utilize the NBD RA opportunity plus other related standard Big Data architectures to establish best practice implementation guide through actual testbed development of various unique Big Data use case scenarios using available best technologies and solutions.
  1. Initial Membership


Confirmed members:

First Name

Last Name

Organization

Country

Wo

Chang

National Institute of Standards and Technology

US

Geoffrey

Fox

Indiana University

US

Yuri

Demchenko

University of Amsterdam

Netherlands

Rahul

Ramachandran

University of Alabama

US

Morris

Riedel

Juelich Supercomputing Centre

Germany

Peter

Baumann

Jacobs University

Germany

Andrew

Treloar

??

Australia

Ajit

Gaddam

Visa

US

Arnab

Roy

Fujitsu

US

Beth

Plale

Indiana University

US

Buaey Qui

Ho

Inforcomm Development Authority of Singapore (IDA)

Singapore

Chaitan

Baru

UC San Diego

US

Chen

Xu

George Mason University

US

Chris

Mattmann

NASA

US

Cynthia

Hudson

Washington University in St. Louis

US

David

Boyd

Data Tatics

US

Giuseppe

Flameni

CINECA SuperComputing Center

Italy

Jay

Greenberg

Boeing

US

Jian

Li

IBM

US

Joseph

Fargnoli

Ritre

US

Laurie

Rokke

NOAA

US

Nancy

Grady

SAIC

US

Onur

Savas

Intelligent Automation, Inc.

US

Orest

Swystun

HP

US

Orit

Levine

Microsoft

US

Pavithra

Kenjige

??

US

Pratik

Thakkar

Philips

US

Protima

Banerjee

Lockheed-Martin

US

Pw

Carey

Compliance Partners

US

Renata

Spinks

Treasury

US

Robert

Whetsel

Defense Information Systems Agency (DISA)

US

Robert

Marcus

ET-Strategies

US

Sanjay

Mishra

Verizon

US

Talapady

Bhat

NIST

US

Tim

Zimmerlin

??

US

Weihua

Lu

Huawei Technologies

CN

William

Miller

MaCT

US


  1. References


  1. NIST Big Data Architecture Models: A Survey, M0151

  2. NIST Big Data Reference Architecture, M0266

  3. Apache Big Data Stack (Big Top): http://bigtop.apache.org

  4. Berkeley Data Analytics Stack: https://amplab.cs.berkeley.edu/software

  5. Ubuntu JuJu: http://juju.ubuntu.com

  6. Dell Crowbar: http://www.dell.com/learn/us/en/555/cloud-computing/crowbar-software-framework

  7. Puppet Labs: https://puppetlabs.com






The database is protected by copyright ©essaydocs.org 2016
send message

    Main page