Data API Web Application

Goal
Opening the eRegulations data through REST web services.

Integration within the eRegulations Global System
It must be considered as a satellite of the eRegulations Admin web application (henceforth eRegulations-Admin-App). Even though it is self manageable, it does rely on the eRegulations-Admin-App for data population.

Application general solution
The solution is build on node.js using restify for the REST requests handling and mongodb for data persistence. Each action on the eRegulations-Admin-App that has the quality of delivering public information will trigger the correspondant request (POST/PUT/DELETE) to the API Server for assuring the syncronization of data. All requests are passed through basic authorization filter (username & password) before being handled.

1. Setup

Needed software

Application can be deployed on both OSX/*nix and Windows systems. It needs following software to be installed:

Environment variable

PORT num - (optional) Port on which application main server should be deployed; if not provided then port 8080 is used

Application specific settings

The application specific settings file is config/index.js. Fill the file with the following options :

  • appPath string - holds the application path in case the api is executed as a sub-application inside a more general application; leave blank in case of root application
  • db - mongodb configuration
  • host string - database host
  • port num - database port
  • username string - (optional) User name (if we're using authentication)
  • password string - (optional) Password (if we're using authentication)
  • systemCollectionPrefix string - the common prefix for each of the system dbs
  • systemsCollection string - the name of db collection holding all the eRegulations systems that expose their data through API
  • apiUsersRepository - configure the name of db and collection where all the users being authorized to consume the API
  • dbName string - name of database collection
  • collectionName string - name of collection holding all the users

2. Application

Third party libraries

All modules are placed in the node_modules folder :

restify - https://github.com/mcavage/node-restify
restify used for building REST web services
mongodb - https://github.com/mongodb/node-mongodb-native
MongoDB native driver
mongojs - https://github.com/gett/mongojs
wrapper for the mongodb-native, that emulates the official mongodb API as much as possible
underscore - https://github.com/documentcloud/underscore
JavaScript's functional programming helper library
async - https://github.com/caolan/async
Flow control helper library

3. Application architecture

Integration within the eRegulations Global System

See in the diagram below the two major roles played by eRegulations API server :

  • responding to data requests
  • keeping a synchronized version of eRegulations data by handling all the updates triggered by data management application

NewImage

Directory structure

- app
    |__ controllers/
    |__ routes/
    |__ server.js
- config
    |__ index.js
- middletier
    |__ businessObjects/
    |__ db/
    |__ modelPrettifiers/
    |__ util/   
- app.js
- gruntfile.js
- package.json

Application code

The design of eRegulations API server follows the principles of RESTful way. From Wikipedia :

  • Application state and functionality are divided into resources
  • Every resource is uniquely addressable using a universal syntax for use in hypermedia links
  • All resources share a uniform interface for the transfer of state between client and server

    • The user progresses through an application by selecting links (state transitions), resulting in the next page (representing the next state of the application) being transferred to the user and rendered for their use

    The application follows the 3-tier architecture, grouping the presentation layer code in the app folder and the business logic in the middletier folder.

    Upon receiving a request the server does the following actions:

    1. authenticate & authorize the request by reading the authorization header. The server implements Http Basic Auth strategy.
    2. if user is authorized, then the server delegates to a specific handler based on the resource identifier and the verb in the header of the request.

    app

    implementation of API REST server (request handling)

    • controllers/ - all the handlers for a specific resource are grouped inside a controller
    • routes/ - holds all the url patterns and also the mappings for a specific action on a resource

    middletier

    communication with the database, exposing business objects implementing different calculations on top of models.

    • businessObjects/ - objects responsable for database operations on the specific model, the pre-processing of data before presistence or altering right after retrieval from db
    • db - enum of collections, common CRUD db operations and the management of db connections
    • modelPrettifier/ - prepares the model prior to delivery: removes inner data from the model (ids, attributes, etc), aggregates data into a readable format (the schedule info)
    • util - generic utility functions

    Feedback