Challenge Overview

You are not writing or developing nodejs application in this challenge. You are writing shell scripts for a nodejs application.

Project Overview

The CSFV Community Gaming website project will provide the web site infrastructure and integration of games built to make formal program verification more cost-effective. This is web site that is available on the Internet http://www.verigames.com, and also delivered in a form where the server and the games can be installed and used behind a firewall.

Challenge Overview

We want to simplify the process of setting up development environment for competitors, and reviewers. The goal of this challenge is to produce shell scripts to automate deployment of CSFV applications from github repositories, automating running the applications, and automating the process to generate git patches and applying it.

Challenge Requirements

You will write shell scripts that address the following tasks :

  1. Cleanup task :
    1. This task will cleanup all the folders created by Checkout and Deploy tasks
  2. Checkout task :
    1. For each github repository (the repositories will be configured and will be executed in order):
      1. Checkout the code from configured repo link, by default, the folder name will match the repo name if 'folder_name' attribute is not configured
      2. Checkout the branch configured in 'branch' attribute, if not set, then checkout 'master' branch by defualt.
      3. Create local branch of checked out branch from step (2), name of branch will be same as the checked out branch prefixed with "local_" string.
      4. (optional step) Lookup the patch name that matches the checked out repo folder name (with .patch extension) and apply it to the created local branch in step (3).
      5. Run 'npm install <dependency>' where dependency is read from a configured list of checked out repositories that this repo depends on. i.e. csfv_cms_module depends on csfv_core_module, the command will be 'npm install ../csfv_core_module"
      6. Run 'npm install' , you might need to pass arguments to this command so the result of executing it won't override the modules we installed in step (5)
      7. cd .. to root folder.
  3. Npm install task :
    1. This task will clean up the folders to be replaced in node_modules of repositories and run 'npm install {dependency}' as outlined in Checkout task.
    2. User can optionally run this task against specific checked out module. i.e. to recreate node_modules under csfv_frontend_module I should be able to run './deploy.sh --npm-install csfv_frontend_module'
    3. It should also have option to install only the checked out folders or specific ones. We don't need to checkout external 3rd party libraries each time.
  4. Run task :
    1. Validate that memcached and mongo db are running.
    2. Setup entries in hosts file.
      • It is simple line 127.0.0.1 csfv.com
    3. Copy env sample file to .env
    4. For each configured application that can run as standalone app :
      1. Chose an arbitrary unused port in the Registered Port range - from 1024 through 49151 - and assign to the app by configuring the env variable used by the app.
      2. Store the port used in a folder created in root folder, the file shoud be named {folder_name}_info file.
      3. Run 'forever' command of each start_files entry and append the PID and Log file name in {folder_name}_info file used in step (2)
        • It is not your responsibility to ensure that the application started successfully in forever. There can be errors caused by wrong configuration. It is out of scope.
      4. Log errors or successful execution and print the {folder_name}_info content to log/stdout.
  5. Apply patch task :
    1. It is It applies patches on each repo folder.
    2. It should first validate the patch then apply it.
    3. You can refer to this post for more info https://ariejan.net/2009/10/26/how-to-create-and-apply-a-patch-with-git/
  6. Generat patch task :
    1. This will generate patch files of each configured repository
    2. It should simply run the command that generate all commits made locally git format-patch master --stdout > {folder_name}.patch
    3. Zip the patches in a zip file with name csfv_submission_{timestamp}.zip
    4. Cleanup the patch files after zip generation
    5. Ask the user to upload submission to Online Review =)

Note that user should be able to run multiple tasks at the same time, or run a single task at a time. Generating patch task should NOT be be part of the task that run them all, it will not sound right to include it :)

Global Bash Functions

We want to study the possiblity to define the tasks as bash functions in .bashrc file so members can execute it from any folder without the need to copy/paste the deploy file to that directory.

Configuration File

There should be a configuration file that contains following information :

  • List of repositories to be checked out
    • Git url : represents the URL used to checkout the repo
    • branch (optional) : can be branch name, commit id, or tag. Script will use 'master' if this is not set.
    • folder_name (optional) : represents the folder name to checkout the repo to. It will be same repo name if absent.
    • dependencies (optional) :
      • Array of the checked out repositories names this repo depends on. The dependency name should match 'folder_name' if provided in that dependency's config.
    • standalone (optional) : optional flag indicates whether this app can be executed by Run task as standlone app.
    • startup_files (optiona) : an array of js files, this must be provided if standalone is true, it will be the js file names to be executed by forever command.
    • .. Other attributes if needed
  • Log folder path : where log files will be stored
  • Info folder path : where the info files will be stored
  • Version : script version
  • .. other configurations if needed

General Notes

  1. Every single step should be logged to stdout and a configured log file. Exit when error or when successfully done should be clearly logged as well.
  2. Shell script can prompt user for input if needed input from user.
  3. The shell usage should be clear and user should be able to ask for it. i.e. ./script.sh --help would print something like (this is example and you don't need to use same options names/alias) :
    Usage ./script.sh [OPTIONS]
    -c, --clean      Clean up the repo folders
    -d, --deploy    Run the deployment scripts
    .. etc
  4. For package.json of repositories that include csfv_xxx modules under 'dependencies' attribute, you need to report that in challenge forums and Copilot will fix repo to move them to 'devDependencies' folder.
  5. The tasks should be written taking into account that user can rerun it many times, it should always work as expected.

Github Repository

The code exists in private github repository https://github.com/topcoderinc/

This URL will give list of CSFV modules in scope https://github.com/topcoderinc?query=csfv_+only%3Aprivate+only%3Asources+ excluding the following modules :

  • csfv_logaholic
  • csfv_blog_module
  • csfv_ui_prototypes

Also nodejs_sitemap_generator is in scope nodejs_sitemaps_generator

The application that can run as standalone are :

  • csfv_frontend_module
  • csfv_minisite_frontend_module :
    • This repo should be checkedout 5 times
    • Each folder :
      • branch : minisiteX where X 1-5
      • folder_name : minisiteX where X 1-5
  • csfv_test_data_generator
  • csfv_chat_module
  • csfv_notification_module
  • csfv_chat_frontend_app
  • csfv_oauth2_server_module
  • csfv_gaming_api_module
  • csfv_cms_module
  • nodejs_sitemaps_generator

Lookup dependencies of each CSFV module from package.json

Target Environment

The script should run in *nux based OS (including Mac).

Project VMs

VMs will only be granted to experienced members. You have a past record in TC challenges. Or you can email Copilot with good reason/reference to assign you a VM if you are new to TopCoder challenges :)

Members will assigned a CentOS VM that have Memcached and mongo db installed.

 



Final Submission Guidelines

Submission Deliverables

Please refer to the Assembly Tutorial for submission deliverables: http://apps.topcoder.com/wiki/display/tc/Module+Assembly+Competition+Tutorial

  • Submission archive containing 1 deploy shell script and 1 deploy configuration file that addresses all tasks/requirements outlined above.
  • Deployment guide containing step-by-step instructions for deployment.

ELIGIBLE EVENTS:

2015 topcoder Open

REVIEW STYLE:

Final Review:

Community Review Board

Approval:

User Sign-Off

SHARE:

ID: 30046651