Collaborate, Create, Communicate

Giving Research New Bearings: Take a Peek in the Google Engine Behind CGMap

CGMap, a System-wide application that enables users to navigate easily through information on research and research-related activities that the CGIAR Centers and Challenge Programs publish in their Medium Term Plans (MTPs) every year, recently teamed up with the CGIAR’s Regional Plan for Collective Action.  Their goal?  To put active research projects in East and Southern Africa firmly on the map.

Read on for a rare glimpse into the workings of the CGMap application. Although this article is a little technical, we’ve tried to keep it as simple as possible, in the hope that it will guide you towards a better understanding of what goes on in CGMap’s navigation room.

Putting Research on the Map: Pushing the Envelope of Google Data APIs

Increasing the impact of agricultural research is a shared interest of the CGIAR Centers and their partners.  In 2007 and 2008, the CGIAR’s Regional Plan for Collective Action surveyed and collected information from the Centers and their partners on research projects being carried out in Eastern and Southern Africa (ESA), a key geographical region for agricultural research.

Research on the Map

In 2008, the Regional Plan for Collective Action and the ICT-KM Program joined forces to realize the next natural step for the Plan: putting active research projects in ESA on the Map.

The ICT-KM Program promotes initiatives that accelerate the CGIAR’s efforts to generate, safeguard and share knowledge in new ways.  In fact, the Program’s 2008 launch of CGMap revolutionized the access to and the analysis of CGIAR Medium Term Plans (MTPs) and their related data (financial, geographic, topic) across CGIAR Centers and Challenge Programs.

Beyond Boundaries: Google Data to a Database and Records Management System

CGMap established the precedent for the contribution and diffusion of CGIAR research.  At the ICT-KM Program, our challenge with ESA was to deliver a cost-effective, quick-to-market system to facilitate the contribution of the ESA research projects from 2009 and beyond, and put them on the map.

We’d already had vast experience with the Google Apps Case Study, a joint initiative with the CGIAR IT Managers, as well as Google Maps and Google Search on CGMap.  As such, we were aware that the Google Data APIs provide simple, standard syndication protocols for reading and writing data/information/content.

However, the Google Data APIs, particularly the service-specific Google Spreadsheet API with Forms, do not provide relational databases.  Nonetheless, after a brief study of the application’s documentation, we knew that we could make a simple and technology-neutral system with relational data for both contributing and mapping the research projects.

How we did it

Defining Data Records
We identified and defined the information to be captured, as well as the record classification for identifying unique records and tracking the latest revisions.We then created a spreadsheet in Google Docs, with the first sheet containing the required information structure.Our model can store records and the history of records. As such, a complete revision history is always stored in the system.
Data Records
Defining the Data Elements
A second sheet was created to hold a definition of the data elements stored in the data records sheet.  The following elements were defined: text, check lists, select lists, and data fields.  We also defined ‘system’ configuration fields for logical/automated fields.This data element definition sheet allows us to identify aspects of the data, such as, “is the field required?”,  “in what order should the data appear?”, or “does the information stored in the field come from an external source, and if so, which one?”.For example, the Countries field is:

  • required
  • checklist
  • and the values are stored and defined in a separate source (a controlled list)
Data Definition
Defining Controlled Lists
In a separate spreadsheet, we defined sheets to contain controlled lists for Countries (with geo-center point latitude and longitude, ISO and UN codes), CGIAR Centers, Partner Types, MTP Projects Lists (from CGMap with the CGIAR Center acronym), CGIAR Regions, as well as a sub-list of African Countries.
Controlled Lists
Providing a Simple Interface to Contribute Data
Building on the contribution mechanism provided by CGMap and taking advantage of Google Data APIs, we built an HTML form using the Google JavaScript libraries (JSAPI) to access the data record element definition (see Defining the Data Elements above). This rich and interactive form is presented to the contributor with all the required fields and values based on the data record element definition. We take advantage of the Google Data APIs’ mashup components to utilize the Yahoo Interface Libraries (for calendar date selectors) and our own custom scripts for checking the integrity of the information provided by the contributor.Saving the record submits the information, via the Google Spreadsheets Forms API, to the data record spreadsheet.
Putting the Research on the Map
Again, we took advantage of CGMap’s ability to navigate research projects via a map interface.A map is built using the Google Maps API, then we query the data records to obtain the latest records’ country field, which contains ISO Alpha-3 country codes.  The center point of each country with a record is plotted with a map marker by looking up the country’s coordinates in the Countries controlled list spreadsheet.When clicked, the map marker opens an embedded page (the project list bubble).  This page recognizes the currently open country map marker, and then queries the data records for those projects in that country.The project list bubble displays the project title with a link to display the data record as a fact sheet.  It also utilizes the Google Charts API to build a contextual map that provides a geographic visual overview of the project.The fact sheet queries the data records for all fields in the requested record.  A larger contextual map is then displayed, and based on the record elements definition, each field is displayed in the sheet.  When a record contains MTP project relationship data, the Google Search API is utilized to search CGMap to display relevant links to CGMap Project Factsheets.
Map Navigation


Take a peek at the ESA Research Maps!