Key Information

Register
Submit
The challenge is finished.

Challenge Overview

Project Overview

The Gmail API Search and Delete Tool is an administrative tool that can search for and delete messages in a Google Apps user’s Gmail account.  This application will also be responsible for OU based authorization and logging of applications actions.

Competition Task Overview

The Gmail API Search and Delete (SAD) Tool has been developed but needs to be restructured for scalability.  Currently the application will run if there are hundreds of users in a Google Apps Domain, but currently will fail if there are thousands. There are 2 main issues we have found that must be resolved. 

  1. User List Creation Look Up Optimization
  2. Message Retrieval Converted to Task Queues

If you notice others, feel free to address them as you see fit. Bonus points for other changes that you make to improve efficiency and call out in your guide documentation.

It is essential that all existing functionality is maintained and optimised.

Following information points to areas that our team have identified as problem areas and is not meant to be a step by step process for addressing all performance and scaling issues. 

1. User List Creation Look Up Optimization 

The user list is a provided CSV file but needs to be sanitized and each user needs to be verified to be within the authorized user's OU and sub-OUs. This is currently done but the way in which it has been coded is not the most efficient for more than 500 users.

  • File: UserActionServiceImpl.java
    • Function: public List<UserAction> searchUserActions(String searchTerm, String userEmail)

The function above and possibly others takes the list of provided users loops through the list and gets each user independently to check their OU against the authorized user's OU and sub-OUs. A more efficient way would be to:

  1. Get user list by OUs (Authorized user's OU and sub-OUs)
  2. Use above mega list of users to compare with the provided CSV.
  3. In the case of the special "catch-all" use the generated list of users in step 1.

1.1 User List Creation Look Up Queue

Currently the searchUserActions function is called from the AJAX (script.js) as part of a set of nested calls. The AJAX call to "/userActions" is mapped in  HomeController.java to searchUserActions, as described in the previous requirement. In order to avoid timeout issues within App Engine the AJAX call to "/userActions" should be making a call to help function that will enqueue a searchUserActions call.

1.2 Modify List Creation to Begin /userActions/id/execute

Currently once the userActions command has returned, another nested AJAX call is made to /userActions/id/execute. This action is dependent on the results of the previous call to /userActions. Because of this, the last step in the successful completion of /userActions should be calling  /userActions/id/execute

1.3 Action Execution Queue

Currently /userActions/id/execute maps to the executeUserAction in  UserActionServiceImpl.java. This function does a number of things before calling a second function processEmailActions in UserActionServiceImpl.java. Within processEmailActions is a for loop that must be refactored so that each iteration of the loop can be added to Google task queue*

Additional refactoring of the function to deal with the asynchronous return of the data will need to happen.

2 Search and Delete Queues

Within processEmailActions loops one of two functions is called: searchMessages or searchAndDeleteMessages both from EmailMessageServiceImpl.java. Both functions will require the implementation of Google task queues*.

  1. searchMessages: the function contains two loops. The while loop returns pages of messages IDs that match the search. After each page of messages the tool makes a call to the Gmail API to retrieve the individual messages, this code is contained within a for loop. The for loop calls fetchEmailMessage and appends the message details to an array of emailMessages. The code should be modified so that each email that must be returned is added to a queue so that the code can continue and page through additional results. This will require refactoring of how the messages are eventually stored.
    • Side note: add a flag to the creation of these tasks for future functionality. The flag will allow the code to decide to get the message details or not.
  2. searchAndDeletesMessages: this function contains a single loop. A single call is made for each cycle through the loop. This will need to be refactor so that each deletion call can be enqueued instead of synchronously completed. Note that this function also calls searchMessages, this may impact how you refactor the searchMessages code.

* Google Task Queue Note: Use the processing rates to manage the queue to keep the API limits happy.

Testing

The appirio-dev3.com domain has ~2000 users now and you can use this domain in development phase.

The winning submission will be tested in another domain which has more users. Winner will be granted access to that domain and should make sure the final fix submission code works in that domain.

Technology Overview



Final Submission Guidelines

  • Deployment Guide
  • Source Code
  • Deployed App Engine Project

ELIGIBLE EVENTS:

2015 topcoder Open

REVIEW STYLE:

Final Review:

Community Review Board

Approval:

User Sign-Off

SHARE:

ID: 30048490