CMAR c-squares mapper and the c-squares Xplanet mapper README file

Accompanies Mapper Version: 3.0
Last Updated: 24 May 2006
Copyright under GNU General Public License (GPL) : CSIRO Marine and Atmospheric Research (CMAR), Hobart, TAS 7001, Australia. See included file disclaimer.html for further information on usage
README file authors: Philip Bohm and Tony Rees, CMAR.
README file location (check occasionally for updates):
http://www.marine.csiro.au/csquares/mapper_README.html.

Description

The c-squares mapper (cs_map.pl referred to hereafter as the mapper) is a web-accessible, perl based utility (application) which creates web plots of c-squares (typically, representations of dataset spatial extents or "footprints") on any of a range of user-selectable, pre-prepared base maps, according to one or more strings of c-square codes passed to it via the web, either by a human user or (more typically), a remote application.

The c-squares xplanet mapper (csqXplanet.pl referred to hereafter as the Xmapper) takes large global maps created by the mapper in Mercator projection and wraps them into the orthographic projection. The mapper must be run first.

More general information about c-squares principles, notation, etc. can be found at http://www.marine.csiro.au/csquares/.

Version 1 of the c-squares mapper was created in January 2002 by Tony Rees and Miroslaw Ryba to perform mapping functions for various web-based CMAR applications, including MarLIN, CAAB and others (for further examples refer http://www.marine.csiro.au/csquares/examples.htm), and has since been used in a similar manner by a number of applications across the world.

Version 2. The original version of the mapper was upgraded in 2003 by Philip Bohm and Tony Rees to provide a significant range of new functions including active (clickable) maps, support for multiple c-squares strings, decompression of c-squares passed in "compressed" notation, and more, and has since undergone various more minor revisions to provide additional or improved functionality.

Version 3 creates extra maps behind the scenes when a user selects a full size global image. The mapper then provides links to the csquare Xplanet mapper which uses the extra maps to create global views. The basemaps may now be either gif, png or jpg formats. See: http://www.obis.org.au/admin/csquares/help.html

Installations of the c-squares mappers at CMAR currently run on UNIX and SUSE Linux and the mapper and Xmappaer have also been successfully installed and run on Windows servers.

For a reference installation of the mapper, visit http://www.obis.org.au/cgi-bin/cs_map.pl. Other current known installations are at http://filaman.uni-kiel.de/cgi-bin/cs_map.pl. The original version of the mapper at http://www.marine.csiro.au/cgi-bin/cs_map.pl has been discontinued and now redirects transparently to the installation on obis.org.au.

A screenshot of a (fairly sophisticated) example map created by a client application is available here. This map (created by the European AquaMaps project) exercises a number of features of the mapper including plotting filled as opposed to outline squares, plotting multiple c-square stings in a range of user-defined colours, including custom graphics and HTML formatting in the supplied header, title and legend, and including clickable maps (in the online version) that return spatially relevant information regarding the point on the map where the user has clicked. Examples of other types of maps that can be created can be located via the c-squares website, http://www.marine.csiro.au/csquares/.

Mapper Features

• Accepts one, or multiple (up to seven) c-square strings for rendering on appropriate base maps. Each string can comprise one to an unlimited number (limited in the software to less than 1 million) of square codes, separated by the vertical bar or pipe (|) character (example: 1000|1001|1002 etc.). Each string can be rendered in a separate colour or can be a different resolution (square size) from other strings; however, all codes within a string must be the same resolution and will be plotted identically.
  
  
• Squares can be any from the available set of c-square resolutions, i.e. 10 degree, 5 degree, 1 degree, half degree, 0.1 degree, etc. The resolution (square size) is represented by the length of the individual c-square code e.g. 1000 (10 degree square), 1000:1 (5 degree square), 1000:100 (1 degree square), 1000:100:1 (half degree square), etc.
  
  
• Colours for individual c-squares strings are user definable (hex) colours, e.g. FF00FF (will give a purple square), FF9900 (will give an orange square). If no user definitions are sent, a set of default colours (red through brown) will be used.
  
  
• C-squares may be sent to the mapper in a compressed state and are decompressed by the mapper. Example: 1000:* may be sent instead of the four squares 1000:1 through 1000:4 ; 1000:*** may be sent instead of the one hundred squares 1000:100 through 1000:499 (saving transmission time and bandwidth, since all squares are sent to the mapper in the first instance, and are also returned along with the map to enable subsequent requests e.g. for different choice or size of base map etc.)
  
  
• When called by HTTP "POST" (recommended method), the mapper will potentially cope with unlimited numbers of squares (has been tested with upwards of 100,000 squares without problem on a suitable fast server). The only limitation may be the timeout settings of the host machine, web gateway, or browser client if processing is slow.
  
  
• The c-square codes are checked for validity and the mapper will return informative error messages if a bad code or string segment is detected.
  
  
• The mapper will by default render the supplied squares on a "best fit" base map (with user option to change to other maps as desired), however this can be overridden if the client supplies the file name of the desired map for rendering.
  Note: previous to version 3 the name of map as viewed in the drop down box was used.
  
  
• Maps can be drawn in two sizes, "normal" (suggested up to 600 pixels wide) and "large". Large maps may be drawn from very large base maps, but are resized and returned to the user as 900px maximum in cs_map.pl. The large base maps are retained for use in csXplanet.pl where resolution while zooming is important. The default size is "normal", however the user or remote application can request a large size starting map if desired. The global views produced by the Xmapper can only be created from large global maps identified with filenames such as '<fileidentifier>-7200<file_extension>'. eg worldmap-7200.gif
  
  
• Land masking - where supported, portions of plotted squares on land will be masked out when this feature is activated. (E.g., useful for distribution maps of marine-only phenomena, to avoid overwriting of the coastline). This facility is not currently available for every base map.
  
  
• If supplied with a "redirect" URL, the mapper will create an "active" (clickable) map. Clicking on the active map will open a new window which is directed to the previously supplied redirect URL plus the variables N_lat, S_lat, W_long and E_long, which form a bounding box at the input resolution determined by the first c-square sent, and can be further handled by a suitable script created by the remote application developer. See: http://www.marine.csiro.au/csquares/mapper-tech.htm. This new window can be requested either as full screen (default) or as a small "pop-up" window.
• The map images themselves can be tagged by inserting a user supplied custom identifier if desired, to assist automated harvesting and collation.
• Individual c-squares can be rendered as a rectangle (default) or a solid coloured ("filled") square, upon request. This feature can be applied to different supplied c-square strings independently.
  
• The default behaviour of the mapper to enlarge the smallest squares to improve readability, also to create a 1 pixel border between adjacent squares; can be switched off to depict a more accurate representation. Large maps greater than 7000px will be drawn with oversized and maybe overlapping csquares to enable visibility unless switched off.
  
  
• Supports user supplied map title, header, and legend which will be rendered above and below the map as appropriate (may be HTML).
  
  
• The variable 'header' can contain HTML and is rendered at the top of the page and also in the browser title. The mapper strips HTML tags from the variable 'header', inserting only the remaining text into the HTML head->title tag.
  
  
• CSS style sheets are now in place for printing and display.
  
  
• Javascript functions are included for the popup window creation and Flash enabled browser detection etc.
  
  
• Although the mapper is most suitably called via HTTP POST for application-level use (supports spaces in variable text, long variables, etc.,), however it can also be called via a direct URL with embedded variables for testing and simple mapping actions.
  
  
• Correctly handles Pacific centred world maps (i.e., with borders at 0 degrees E/W as opposed to 180 degrees E/W), if these are selected.
  
  
• Map creation time, plus number of squares processed, is automatically captured and displayed with each map (useful for monitoring system performance, etc.).
  
  
• Can be used to generate inline maps (returns a graphic only, without surrounding HTML wrapper) on request, for embedding into a client application's web page.
  
  
• The size and position of popup windows (used for "click on a map" functionality) are user specifiable.
  
  
• The "click on a map" function can be engaged without requiring the mapper to create any maps, making the application suitable for processing of many standalone spatial search interfaces for web use (for examples, see http://www.marine.csiro.au/marlin/csq-chooser.htm and http://www.obis.org.au/data_search/mapsearch.html).

Xmapper Features

See: http://www.obis.org.au/admin/csquares/help.html

Contents of the installation package

For distribution, the mapper/Xmapper version last update date is appended to the file name (example: cs_map2005111.pl), in which case the file should be renamed to cs_map.pl and csqXplanet.pl before installation, by stripping the appended date. Other included files support the mapper and are required to produce the maps and aid with web presentation.

The following lists the required files. Other files may be included but are most likely not required.

In the folder csquares ->

cs_map[date].pl (latest version of the cs_map.pl script. Expects csqXplanet.pl to be present)
csqXplanet[date].pl (latest version of the csquares Xplanet mapper script. Designed to be part of this package)
csqlogo.gif
csqlogo.swf
csquare.css
c-square.ico
csquare_print.css
disclaimer.html
funcs.js
mapper_README.html
maps (folder)

In the folder csquares/maps ->

Hardware Requirements

1. A web server capable of running CGI programs (web viewable) Windows or Linux.

2. A basic server with minimum resources may not give satisfactory service when requesting large maps or when large amounts c-squares are to be processed. For example, 90,000 squares are currently rendered in 23 seconds on our reference, dual 3.2 Ghz processor Linux server. Rendering the same number of squares on our original 167Mhz Unix machine would take about 10 minutes. Large global basemaps maps from '-7200' files are 7200 pixels wide and these are resized up to three times for returning to the user in the mapper and for later use by the Xmapper. The resizing is processor intensive and may account for 10 seconds of processing time even on a fast machine.
3. When called by HTTP "POST" (recommended method), the mapper will potentially cope with unlimited numbers of squares. The only limitation may be the timeout settings of the host machine, web gateway, or browser client if processing is slow.
4. Ideally the server (and client) will need to connect to the internet via a high speed link. The HTML file size is around 1 Megabyte when the combined c-square strings total 100,000 squares (although many requests would be typically much smaller than this).

Required software

1. Perl
2. GD package for perl
   -Note: The GD we're referring to is the GD Perl Module, or GD.PM. There is also something else often referred to as GD: the GD library file (or libgd) which is a language independent graphics library (and is used by GD.PM). That library is not useful by itself.
3. CGI package for perl
4. Xplanet executable

Installation procedure for Linux

The executable file that creates the initial maps is cs_map[ date created ].pl, which is a Perl file. It must be first be renamed to cs_map.pl.

The executable file that creates the global maps is csXplanet[ date created ].pl. It must be first be renamed to csXplanet.pl. All other included files support the mapper, which are simply served by the hosting web server.

1. Ensure Perl and the GD and CGI packages are installed on the web server
   GD may no be packaged by default. If this is a new install I have found ActivePerl to be very easy to add packages to. The mapper needs the GD Perl wrapper installed as a module of Perl.
2. Install Xplanet via readme in package with the following alterations to the source code so that the default shading of the globe will be suppressed. Xplanet will become a executable program in the path of all users after using configure and make. On linux if the intall folder location is not in the right Path the whole folder can be moved.
   
   Changes to make:
   in drawProjection.cpp and libmultiple/drawSphere.cpp
   
   In drawProjection.cpp, at line 127:
   const bool limbDarkening = (options->Projection() == HEMISPHERE || options->Projection() == ORTHOGRAPHIC);
   
   Set limbDarkening to false.
   
   const bool limbDarkening = false;
   
   in libmultiple/drawSphere.cpp, at line 102, there is this block of code:
   for (int k = 0; k < 3; k++) color[k] = static_cast<unsigned char> (color[k] * darkening);
   You can get rid of that block, and then the color won't be affected by limb darkening.

1. unzip csquares.zip into a web accessible directory folder called 'csquares'
2. rename and move the file cs_map.pl and csqXplanet.pl into a directory where CGI programs can execute
   (this is usually the folder named 'cgi-bin')
3. All other files should live in a folder that you created called 'csquares' inside your main 'public_html', 'htdocs' or 'www' folder used by your web server.
4. If desired, edit the content of the file disclaimer.html to include your agency's name and acronym in the appropriate places.
5. Edit the variables that describe the file paths to the file cs_map.pl. In cs_map.pl the paths are set for a linux installation. In our cs_map.pl file the variable $BASE_MAPS the path is: /srv/www/htdocs/csquares/maps. You will need to change the /srv/www/htdocs/ part to reflect the path to the installed folder 'maps' for example.
   
   Here is the appearance of the relevant portion of cs_map.pl as currently installed at www.obis.org.au on Linux:
   
   

# ------------------------------------------------------------------------------
BEGIN {
 $VERSION = "3";
 $WWW_ADDRS = "http://www.obis.org.au"; # home page (used in credits)
 $URL_2_CGI = "$WWW_ADDRS/cgi-bin/"; # path to this file (and csqXplanet.pl) 
 $FILE_NAME = "cs_mapVer3.pl"; # this file name
 $BASE_MAPS = "/srv/www/htdocs/csquares/maps"; # location of maps and temp map folder on server
 $TEMP_MAPS = "tmp_maps"; # writable folder within $BASE_MAPS
 $MAP_INDEX = "$BASE_MAPS/Map_Index2"; # map index file
 $WEB_FILES = "$WWW_ADDRS/admin/csquares"; # included files - css, gif, swf, js etc
 $TMPMAPURL = "$WEB_FILES/maps/$TEMP_MAPS"; # Absolute url to temp maps
 $CSQR_PAGE = "http://www.marine.csiro.au/csquares/about-csquares.htm"; # c-square docs location 
 $XPLANCONF = "/usr/local/share/xplanet/config/csq.txt" # xplanet (3d Mapping) config file
}
# ------------------------------------------------------------------------------
	

• LINUX ->	$BASE_MAPS = "/srv/www/htdocs/csquares/maps";
  WINDOWS ->	$BASE_MAPS = "f:/wwwroot/filaman.uni-kiel.de/htdocs/csquares/maps";

Modify the first "shebang" line in cs_map.pl and csqXplanet.pl as required

The original line looks like :
#!/usr/bin/perl

you must change it to
#!<where your perl interpreter lives>

Modify the file permissions of cs_map.pl to 755 or similar (owner=rwx group=rx others=rx) if required

Modify the file permissions of the temporary map (tmp_maps) folder to enable the writing of files at run time if required.

To enable (the essential) automatic deletion of old temporary maps via cron create a crontab file as the same user as the owner of the 'tmp_maps' folder

Edit the crontab and install with the following line completed with your timing settings and paths:

Linux example only:

# clean out old cs_map.pl maps
15 * * * * find /srv/www/htdocs/csquares/maps/tmp_maps/ -type f -amin +360 -exec rm -f {} \;
16 * * * * find /srv/www/htdocs/csquares/maps/tmp_maps/ -type d -mmin +360 -exec rm -drv {} \;

High usage of the mappers use a lot of hard disk space. ( 20 uses per hour = 400Mb) Change to suit your requirements.

For Windows Installation

The executable file that creates the initial maps is cs_map[ date created ].pl, which is a Perl file. It must be first be renamed to cs_map.pl.

The executable file that creates the global maps is csXplanet[ date created ].pl. It must be first be renamed to csXplanet.pl. All other included files support the mapper, which are simply served by the hosting web server.

1. Ensure Perl and the GD and CGI packages are installed on the web server
2. GD may no be packaged by default. If this is a new install I have found ActivePerl to be very easy to add packages to. The mapper needs the GD Perl wrapper installed as a module of Perl.
3. Install Xplanet with the following alterations to the source code so that the default shading of the globe will be suppressed. Xplanet will become a executable program in the path of all users after using configure and make. On linux if the intall folder location is not in the right Path the whole folder can be moved.
   
   Changes to make:
   in drawProjection.cpp and libmultiple/drawSphere.cpp
   
   In drawProjection.cpp, at line 127:
   const bool limbDarkening = (options->Projection() == HEMISPHERE || options->Projection() == ORTHOGRAPHIC);
   
   Set limbDarkening to false.
   
   const bool limbDarkening = false;
   
   in libmultiple/drawSphere.cpp, at line 102, there is this block of code:
   for (int k = 0; k < 3; k++) color[k] = static_cast<unsigned char> (color[k] * darkening);
   You can get rid of that block, and then the color won't be affected by limb darkening.

1. unzip csquares.zip into a web accessible directory folder called 'csquares'
2. rename and move the file cs_map.pl and csqXplanet.pl into a directory where CGI programs can execute
   (this is usually the folder named 'cgi-bin')
3. All other files should live in a folder that you created called 'csquares' inside your main 'public_html', 'htdocs' or 'www' folder used by your web server.
4. If desired, edit the content of the file disclaimer.html to include your agency's name and acronym in the appropriate places.
5. Edit the variables that describe the file paths to the file cs_map.pl. In cs_map.pl the paths are set for a linux installation. In our cs_map.pl file the variable $BASE_MAPS the path is: /srv/www/htdocs/csquares/maps. You will need to change the /srv/www/htdocs/ part to reflect the path to the installed folder 'maps' for example see below.
   
   Here is the appearance of the relevant portion of cs_map.pl as currently installed for testing purposes on my Windows machine (probably not running by the time you read this):
   
   

# ------------------------------------------------------------------------------
BEGIN {
 $VERSION = "3";
 $WWW_ADDRS = "http://C000178-hf.hba.marine.csiro.au"; # home page (used in credits)
 $URL_2_CGI = "$WWW_ADDRS/cgi-bin/"; # path to this file (and csqXplanet.pl) 
 $FILE_NAME = "cs_map.pl"; # this file name
 $BASE_MAPS = "C:/csquares/maps";			 # location of maps and temp map folder on server
 $TEMP_MAPS = "tmp_maps" # writable folder within $BASE_MAPS
 $MAP_INDEX = "$BASE_MAPS/Map_Index"; # map index file
 $WEB_FILES = "$WWW_ADDRS/csquares"; # included files - css, gif, swf, js etc
 $TMPMAPURL = "$WEB_FILES/maps/$TEMP_MAPS";	 # Absolute url to temp maps
 $CSQR_PAGE = "http://www.marine.csiro.au/csquares/about-csquares.htm"; # c-square docs location 
 $XPLANCONF = "C:/xplanet/config/csq.txt" # xplanet (3d Mapping) config file
}
# ------------------------------------------------------------------------------

• LINUX ->	$BASE_MAPS = "/srv/www/htdocs/csquares/maps";
  WINDOWS ->	$BASE_MAPS = "f:/wwwroot/filaman.uni-kiel.de/htdocs/csquares/maps";

Change the first line in cs_map.pl and csqXplanet.pl The original line looks like :
#!/usr/bin/perl

You must change it to
#!<where your perl interpreter lives>

for example:
#!c:/usr/bin/perl.exe

this may also work
#!perl

Uncomment the lines "+ binmode ....... " wherever it appears in the perl script

Create a Visual Basic script which deletes all the generated maps. This is more involved than previous versions of the mapper as we now have subfolders. The example below is from a microsoft help page and should work across various Windows variants. Also create a task in the scheduler of windows, so that it gets executed when required.

REMEMBER TO CHANGE THE PATH TO YOUR TEMPORARY IMAGE FOLDER.

' WARNING: This visual basic script will remove the contents of strFolderName
' 
'
'
strFolderName = "c:\csquares\maps\tmp_maps"
Dim arrFolders()
intSize = 0
strComputer = "."
Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
Set colSubfolders = objWMIService.ExecQuery _
 ("Associators of {Win32_Directory.Name='" & strFolderName & "'} " _
 & "Where AssocClass = Win32_Subdirectory " _
 & "ResultRole = PartComponent")
ReDim Preserve arrFolders(intSize)
For Each objFolder in colSubfolders
 GetSubFolders strFolderName
Next
Sub GetSubFolders(strFolderName)
 Set colSubfolders2 = objWMIService.ExecQuery _
 ("Associators of {Win32_Directory.Name='" & strFolderName & "'} " _
 & "Where AssocClass = Win32_Subdirectory " _
 & "ResultRole = PartComponent")
 For Each objFolder2 in colSubfolders2
 strFolderName = objFolder2.Name
 ReDim Preserve arrFolders(intSize)
 arrFolders(intSize) = strFolderName
 intSize = intSize + 1
 GetSubFolders strFolderName
 Next
End Sub
For i = Ubound(arrFolders) to 0 Step -1
 strFolder = arrFolders(i)
 strFolder = Replace(strFolder, "\", "\\")
 Set colFolders = objWMIService.ExecQuery _
 ("Select * from Win32_Directory where Name = '" & strFolder & "'")
 For Each objFolder in colFolders
 errResults = objFolder.Delete
 Next
Next

Testing the installed application

Enter the full absolute URL to the script appending and the minimum required parameter 'csq', 'http://www.yourserver/cgi-bin/cs_map.pl?csq=1000' for example. The mapper requires a valid csq to run or it will return an error message. This in itself is an indication that the script is running of course.

(For Linux only) If your server is a new installation and you want to run a simpler test of the server and external web calls try creating a file called 'test_var' including the code below.

#!/bin/sh
echo Content-type: text/plain
echo
env

Set the permissions!!

The file must have correct line endings for linux. Most PC code editors can save files with unix line breaks

To test Perl If your server is a new installation and you want to run a simpler test of Perl on the server and external web calls try creating a file called 'test_var' including the code below. Remeber to change the path to perl on the first 'shebang' line.

Set the permissions!!

The file must have correct line endings for linux. Most PC code editors can save files with unix line breaks

Trouble shooting (LINUX SPECIFIC)

Customising the Range of Available Base Maps

Individual base maps can be disabled by commenting out the relevant lines in the Map_Index file. Every map should exists in both "normal" and "large" versions, and may also have associated land masks.

Large versions can be identified as '<fileidentifier>[-big or -7200]<file_extension>'. eg worldmap-7200.gif. The Map Index determines priority. The last matching and existing file identified in the Map Index will be the large version. '-7200' in the previous example stands for 7200 pixels wide.

Use the Identifier '-7200' for Large world maps. This identifier tells the software this is a large global map and may be projected into a 3D globe using the csquares Xplanet mapper.

Use the Identifier '-big' for larger size map images of sections of the globe. These images may be any size larger than the 'normal' version.

Note that cs_map.pl resizes all displayed maps at a maximum width of 900px. The proccessing time of cs_map.pl and the size and visibility of rendered csquares is affected by this resizing, so as a rule '-big' images should not be bigger than 900px.

Additional (custom) maps can also be created, however there are a few rules that must be followed in order that the behaviour of the mapper does not become unpredictable (e.g. when selecting the relevant "best fit" map, displaying normal vs. large images in the correct order, etc.). Most of these rules are explained further in the header section of the Map_Index file. Please note, experimentation in this department is at the local installation's risk...

Known Limitations and Bugs

Limitations:

• A hard coded limit is imposed for decompressing any squares of finer resolution than (example) 1000:***:***:* (40,000 sub-squares) since the next value is 1,000,000 which would probably choke the server doing the processing.
  
  
• Squares are plotted sequentially, i.e. csq followed by csq2, csq3, etc. (if present). Earlier plotted squares will be overwritten by later plotted squares if the same square code is sent more than once (i.e., only the last to be sent will be visible - transparency is not supported).
  
• Bugs: All known bugs to the current release date have been fixed. Please report any others found to the address below.

Release and Update Schedule

This application is updated at irregular intervals, depending on user requests and the availability of CMAR staff time for ongoing development. To be informed of the availability of new releases, it is recommended to join the "c-squares-discuss" mailing list (details below).

Feedback and Application Support

Limited support for this application is available on a goodwill basis from CMAR whenever possible. Please address any positive or negative feedback, bug reports, suggestions for improvements, etc. to Tony.Rees@csiro.au. A "c-squares-discuss" listserver is also available (for joining instructions see http://www.marine.csiro.au/csquares/about-csquares.htm#listserver) which interested persons should consider joining, to be informed about (and participate in) ongoing development activities regarding the mapper and the c-squares spatial indexing system in general.