Project Overview

Our client want to use Amazon mturk to invite game players to play set of games. Each Human Intelligence Task (HIT) will be for completing a "task" (analogous to a subset of levels in the game versions) in exchange for payment.

Challenge Overview

In this challenge we are making several changes and update to the mturk web dashboard, and mturk api to support drafting HITs in the dashboard for approval purposes.

Challenge Requirements

The current flow will create Amazon HITs directly when we create a HIT via dashboard or API call (POST {{URL}}/domain/{{domain_id}}/hits).

The new flow will look like this :

• Admin user creates a HIT via api or dashboard

• HIT is created in database with status = draft

• Notification sent to global_admins to notify about new pending HITs with links to these hits.

• global admin will open dashboard, review the HIT, and approve/reject it.

• Admin get notified via email about the approved/rejected HIT.

• Admin user fix the issues by editing HIT or activate the HIT if it is not set to be auto-activated.

• The backend will create the HITs in amazon mturk when activating it.

• Updating HIT will be disabled/disallowed after a HIT being created in amazon mturk (basically status is active and mturkHitId != null where mturkHitId exists in amazon mturk)

• The rest of the flow will be the same.

The changes required to implement the above flow :

• Updates to csfv_mturk_api :

  • Update to allow storing the whole HITs details in Task model when calling (POST {{URL}}/domain/{{domain_id}}/hits)

    • • mturkHitId will equal to NULL since this value is set by the returned HITs id created in Amazon mturk.

    • Add new status field : it should be boolean with values (0/1), 0 indicates it is draft, and 1 indicates it was created in Amazon mturk.

    • Add new step to send notification to global admins about created HITs.

    • Add new flag field (auto-activate) to allow auto creating the HITs in amazon mturk when the HIT is approved.

    • Add new fields to Task model to store information about rejection/approval.

  • Add new endpoint (PUT {{URL}}/domain/{{domain_id}}/hits)

    • The endpoint should :

      • Allow editing Task if it is not created in Amazon mturk yet.

      • Allow storing approval/rejection reason.

      • Allow creating the HITs in amazon mturk.

    • If Task is updated after being approved and before creating in Amazon MTurk then it should go back to review process.

    • It should also accept a flag ‘activate’ to indicate if the HIT should be created in amazon mturk or not.

      • Also if the update was for the approval status, if it approved and the task is auto-create then this step should be executed.

  • GET HITs endpoints should be updated to include the new fields in Task model, user should be able to tell the status of the HIT, review status, auto-create flag, and all HITs fields.

  • Update email notification to include links to the dashboard.

• Changes to csfv_mturk_dashboard :

  • Add New page that list all draft HITs with approve/reject option, and a reason message text field to provide reason for rejection.

    • This page will only be accessible by global admins.

  • In Tasks page, add new drop down to filter Tasks by status.

  • All tables in dashboard : the table pagination is broken, it show buttons to display number of items but there are no buttons to go previous/next, and user cannot tell if there are more items than the currently displayed items. So fix this by adding previous/next if there is more items to view.

  • In create HIT page, update the domain drop down to show names of domains instead of IDs.

    • The same thing for HIT details page, it should display the name of domain instead of the ID.

  • In Edit user popup, it does not show the selected user information, please fix it.

• Add proper validations  in dashboard.

• Update documentation properly, inline comments are great to have.

• Update README.

• Update postman file.

Code Organization

• All logic will be updated and placed in csfv_mturk_api_module

• All frontend changes will be placed in csfv_mturk_dashbaord module

• The code changes in backend should keep each API path/endpoint a self contained api with specific functionality.

• Use helpers for common functionality that will be used by different areas in the code.

Testing

• Make sure to test all possible scenarios, post in forums if you need help to cover them all.

• You will use mturk sandbox for testing.

• You can view created hit by navigation to worker sandbox search for the title of the hit.

• You will need to accept it and manually navigate to the website to simulate the player flow.

• Make sure you can complete the outlined flow above as described. Reviewers are expected to do the same.

Github Repo

API exists in private repositories, request access in challenge forums if you don't have it already.

• https://github.com/topcoderinc/csfv_mturk_api

• https://github.com/topcoderinc/csfv_mturk_dashboard

References

• Mturk Amazon API Documentation

Final Submission Guidelines

Submission Deliverables

Below is an overview of the deliverables:

• Git patch of the changes of each repository.

• Text file with notes to reviewer.

• Update README.md in the API and admin web dashboard git folders, this file should also be part of the patch file.

• Deployment document with detailed verification steps.

Final Submission

For each member, the final submission should be uploaded via the challenge detail page on topcoder.com.