Document Type thesis Author Name Lu, Yue Email Address ylu4 at wpi.edu URN etd-042414-223528 Title CloudNotes: Annotation Management in Cloud-Based Platforms Degree MS Department Computer Science Advisors Mohamed Y. Eltabakh, Advisor Elke A. Rundensteiner, Co-Advisor Craig Wills, Department Head Keywords Annotation Hadoop Date of Presentation/Defense 2014-04-17 Availability restricted
We present an annotation management system for cloud-based platforms, which is called “CloudNotes”. CloudNotes enables the annotation management feature in the scalable Hadoop and MapRedue platforms. In CloudNotes system, every piece of data may have one or more annotations associate with it, and these annotations will be propagated when the data is being transformed through the MapReduce jobs. Such an annotation management system is important for understanding the provenance and quality of data, especially in applications that deal with integration of scientific and biological data at unprecedented scale and complexity. We propose several extensions to the Hadoop platform that allow end-users to add and retrieve annotations seamlessly. Annotations in CloudNotes will be generated, propagated and managed in a distributed manner. We address several challenges that include attaching annotations to data at various granularities in Hadoop, annotating data in flat files with no known schema until query time, and creating and storing the annotations is a distributed fashion. We also present new storage mechanisms and novel indexing techniques that enable adding the annotations in small increments although Hadoop’s file system is optimized for large batch processing.
indicates that a file or directory is accessible from the WPI campus network only.
Browse by Author | Browse by Department | Search all available ETDs
Questions? Email email@example.com