Key Information

Register
Submit
The challenge is finished.

Challenge Overview

This is an application for a data storage, search and retrieval archiving system for emails in Gmail. The key point is to make the system scalable and provide the ability to search potentially huge datasets of emails. The application will provide the ability to archive emails to the cloud from within a Gmail account based upon selection of labels. The application will provide the ability to search through the archived emails on the cloud. The application will provide the ability to restore emails from the cloud back to the user's Gmail inbox.

This module provides the implementation of the backend services of this application. This component provides the implementation of the MailArchiveManagementService, BatchSearchService, BatchSearchWorkerService, CloudStorageService and CryptoService.

Approach: This component acts as a Facade for the archived mails. It encapsulates the communications to different external resources like Solr index, MySQL DB and Cloud Storage.

Indexing And Searching: Solr is used to index the metadata and text of the archived mails. Solr provides a REST API for updating and querying the index. The SolrJ Java Binding provided by Solr is used to send update and search requests. The index update requests are heavy operations, and therefore solr provides multiple update (commit) strategies. From those strategies, the "commitWithin" strategy with disabling "autoCommit" is used in this design. This strategy can be thought of as a server side buffering of update requests. So, it is suitable for the gmail archiver application which expected to have plenty of update requests.

Cloud Storage: The cloud storage used for this application is the Swift OpenStack. Requests to the Swift storage is handled by the jClouds Java Binding. This component provide an abstraction layer (i.e. CloudStorageService) to access the cloud storage. Besides being a Java binding for the Swift API, jClouds is itself an abstraction of multiple cloud solutions like Amazon S3 and Swift. 



Final Submission Guidelines

N/A

ELIGIBLE EVENTS:

2013 TopCoder(R) Open

REVIEW STYLE:

Final Review:

Community Review Board

Approval:

User Sign-Off

SHARE:

ID: 30031760