Coastal Services Center

DTL Project 3: OOSTethys

Overview

The mission of the Data Transport Laboratory (DTL) is to support the employment of data transport technologies that are compatible with Ocean.US Data Management and Communications (DMAC) guidance at the local and regional levels. This is accomplished through the identification, evaluation, and documentation of relevant data transport technology candidates.

In following that mission, the DTL is exploring the OOSTethys implementation of the OGC Sensor Observation Service (SOS). The OOSTethys project consists of members of the ocean-science community who are using Open Geospatial Consortium (OGC) standards to implement an earth-observing "system of systems." OOSTethys partners develop, test and implement easy-to-use, open-source, OGC-compliant software, and have created a working prototype of networked, interoperable, real-time data systems. Details on the OOSTethys implementation and downloads are available at here.

This project has concluded. For detailed findings, see the final report.

• Project Selection
• Related Links
• Server Hardware Details
• OOSTethys Installation and Configuration
• OOSTethys Server Evaluation and Testing

Project Selection

Suggestions for candidate technologies to be implemented in the DTL are accepted from the Integrated Ocean Observing System (IOOS) community. The primary sources for the identification of candidates are the local observatories, regional associations, Ocean.US Data Management and Communications (DMAC) team and its expert teams, and deliberate outreach efforts of the DTL staff, such as meetings and workshops.

Candidate technologies, nominated through these mechanisms, are filtered through a set of criteria to assess their relevance and importance to the stakeholders. For more details, read about the DTL technology selection process.

The selection of the OOSTethys SOS implementation is consistent with the DMAC guidance which espouses a Web Services approach and identifies Open Geospatial Consortium (OGC) specifications as relavent. The OOSTethys SOS service is a Representational State Transfer (REST) based HTTP service.

Criteria	Score	Rationale
Significance to Local Observatories	5	The OGC Sensor Observation Service holds the potential for the development of a service based approach for managing, manipulating, and transporting data from in-situ oceanographic sensors. Further, there is a persistent interest in the local observatory and regional communities in this approach. There are Java and Perl implementations. There is no .Net code base. The technology is portable, scales well, is supported by a wide variety of server and language combinations, and is minimally invasive to the local data repository.
Significance to Regional Associations	5	The project is directly related to the regional and sub-regional arenas of IOOS data transport.
Relevance to DMAC Guidance	4	The OOSTethys work is consistent with the DMAC SOA. The DMAC has identified SOAP/XML as an important research area. The DMAC has also identified OGC WFS as a pre-operational technology. Recent developments in the OOSTethys project have lead the SOAP/XML architecture into an OGC REST/XML direction bring the OOSTethys work in line with several DMAC recommendations.
Relative Maturity of Candidate	5	Data exchange with SOAP or REST with XML are widely supported by many programming languages and server platforms (Apache, IIS, Tomcat, etc.). While there are no fully mature examples of SOAP or REST with XML in the IOOS community, the OOSTethys work is the most fully developed and is a viable option.
Appropriate Level of Effort for DTL	5	The implementation of the OOSTethys servers is appropriate.
Coincidence of other Center Efforts	5	This is a direct result of the Center's Salinity workshop and is consistent with the Center's support of local and regional partners.
Total Score	23

Related Links (top)

OOSTethys Home:	www.oostethys.org/
OGC Home:	www.opengeospatial.org/

Server Details (top)

The OOSTethys software was installed in the DTL on a Dell PowerEdge 1850 machine with two 3.0 MHz Xeon CPUs, four gigabytes of RAM, 300 gigabytes of 10,000 RPM disks, and Gigabit network cards. The operating system is Red Hat Linux Enterprise 4, kernel 2.6.9-42.

OOSTethys Server Installation, Configuration, and Testing (top)

Full documentation for the installation of the OOSTethys server is available from those projects via the links above. The following details the process for installing them on the DTL servers.

Installation

The OOSTethys software was downloaded, reviewed and modified slightly to segregate those portions of the code which were related to a specific installation from those that were generic to all installations. That code is available here. The notes provided here are specific to the DTL implementation of the OOSTethys base code. Please refer to the OOSTethys Cookbook for details on installing their version of the software. The DTL installation was performed as follows.

1. OOSTethys downloads page is here.
2. Dependencies (required Perl modules, available from CPAN)
1. CGI (for handling SOS requests and responses)
2. DateTime::Format::ISO8601 (for parsing and formating dates and times)
3. DBI (for database connection)
4. DBD::Pg (for connecting to PostgreSQL, requires a PostgreSQL installation)
5. File::Basename (for parsing file path)
6. Log::Log4perl (for logging)
7. XML::LibXML (for XML processing)
3. Files:
   1. Modules (reusable)
      - LocalObs.pm Common database API for DTL data loaders
      - LocalObsSosDriver SOS driver for serving sensor information and observed data
      * Read the comments embedded in the source for library reference
      
      
   2. SOS Server 2.1 oostethys_sos.cgi SOS Server CGI (modified to use the SOS driver)
   3. Standard SOS Templates
      - sosGetCapablilties.xml
      - sosDescribeSystem.xml
      - sosGetObservation.xml
      * These files come with the OOSTethys SOS Server package and are not modified
      
      
   4. Logger Configurations for the SOS Server - *.conf
      - Currently only used by the SOS driver and LocalObs.pm.
      
      
   5. Log Files (generated content)
      logs/*.log (current log files for this week)
      logs/old/*.log (old log files for last week)
      
      
4. Installation:
   1. Install the Perl modules listed in the "Dependencies" section.
   2. Make the directory for SOS Server within the cgi-bin directory:
   cd /var/www/cgi-bin
   mkdir -p sos_server/logs/old
   
   3. Copy files listed in the "Files" section to the SOS Server directory and change file permissions:
   cd /var/www/cgi-bin/sos_server
   chmod a+rx *.pm
   chmod a+rx *.cgi
   
   4. Set up a cron job to rotate the log files every Saturday at 23:55:
   55 23 * * 6 mv -f /var/www/cgi-bin/sos_server/logs/*.log /var/www/cgi-bin/sos_server/logs/old/

Base Testing

1. To demonstrate the successful installation of the software, issue some basic requests.

   - getCapabilities
   - describeSensor for sensorID '41001'
   - getObservation
   • Latest sea water temperature for NDBC platform 41001.
     
   • Sea water temperature at 2007-06-18 00:50:00 (UTC) for NDBC platform 41001
   • Sea water temperature between 2007-06-18 00:00:00 and 2007-06-18 23:59:59 (UTC) for NDBC platform 41001
   • Latest wind speed for NDBC platform 41001
   • Wind speed at 2007-06-18 00:50:00 (UTC) for NDBC platform 41001
   • Wind speed between 2007-06-18 00:00:00 and 2007-06-18 23:59:59 (UTC) for NDBC platform 41001

Testing and Critique with External Partners (top)

The DTL solicited participation in the testing and critiquing of this technology from external partners. The external partners will install, configure, and run instances of the OOSTethys server. Objective speed and through-put tests will be performed, and more subjective issues will be discussed and reported. In general, the items to be considered are as follows:

1. Light and heavy load request response times measured with Apache JMeter. The definition of heavy load will be case specific. In general, the test should load the server to the point that requests are queued at the server. Repeat 100 iterations and record each. Calc average, max, and min. The Coastal Services Center will design a basic JMeter test plan that can be refined for each project and implemented.
2. Relative ease or difficulty of installation of server software
3. Relative ease or difficulty of installing and configuring any supporting software or libraries
4. Relative ease or difficulty of modification in the event of evolving data models and standards
5. Degree of intrusiveness of the transport layer into the data storage facility
6. Cost of server implementation (infrastructure and human resources)
7. Cost of client implementations (infrastructure and human resources)
8. Support for server technology in the IOOS community (e.g. continued development, documentation, and user support)
9. Support for server technology in the wider IT community(e.g. continued development, documentation, and user support)
10. Availability of or support for development of clients in the IOOS community
11. Availability of or support for development of clients in the IT community
12. Relative ease of use with conventional or popular client package(s)
13. Degree to which the technology supports IOOS data exchange needs for
    1. Through-put
    2. Ease of use
    3. Discovery
    4. Scalability
    5. Extendibility
    6. Security
    7. Metadata access to determine appropriate use contexts for a given data source.
    8. Mission critical reliability
    9. Data integrity (transactions, error checking missing/redundant/out of range data)

Once the test phase is complete, the DTL will compile the test results and commentary in a report that will be available on the DTL Web site and on the Community Information Repository.