Worcester Polytechnic Institute Electronic Theses and Dissertations Collection

Title page for ETD etd-042414-223528

Document Typethesis
Author NameLu, Yue
TitleCloudNotes: Annotation Management in Cloud-Based Platforms
DepartmentComputer Science
  • Mohamed Y. Eltabakh, Advisor
  • Elke A. Rundensteiner, Co-Advisor
  • Craig Wills, Department Head
  • Keywords
  • Annotation
  • Hadoop
  • Date of Presentation/Defense2014-04-17
    Availability restricted


    We present an annotation management system for cloud-based platforms, which is called “CloudNotes”. CloudNotes enables the annotation management feature in the scalable Hadoop and MapRedue platforms. In CloudNotes system, every piece of data may have one or more annotations associate with it, and these annotations will be propagated when the data is being transformed through the MapReduce jobs. Such an annotation management system is important for understanding the provenance and quality of data, especially in applications that deal with integration of scientific and biological data at unprecedented scale and complexity. We propose several extensions to the Hadoop platform that allow end-users to add and retrieve annotations seamlessly. Annotations in CloudNotes will be generated, propagated and managed in a distributed manner. We address several challenges that include attaching annotations to data at various granularities in Hadoop, annotating data in flat files with no known schema until query time, and creating and storing the annotations is a distributed fashion. We also present new storage mechanisms and novel indexing techniques that enable adding the annotations in small increments although Hadoop’s file system is optimized for large batch processing.

  • (WPI)Master_Thesis_YueLu.pdf

    (WPI) indicates that a file or directory is accessible from the WPI campus network only.

  • Browse by Author | Browse by Department | Search all available ETDs

    [WPI] [Library] [Home] [Top]

    Questions? Email etd-questions@wpi.edu
    Maintained by webmaster@wpi.edu