In this article by Ken Doman, author of the book Mastering ArcGIS Server Development with JavaScript, we will discuss and build applications using the ArcGIS API for JavaScript. We've used different API tools to communicate with ArcGIS Server about its map services. But how does the API communicate with ArcGIS Server?

(For more resources related to this topic, see here.)

In this article, we'll focus on ArcGIS Server. We'll look at how it implements a REST interface. We'll review the ArcGIS REST API, outlined at http://resources.arcgis.com/en/help/arcgis-rest-api/#/The_ArcGIS_REST_API/02r300000054000000/. This describes the file structure and the format of the data that passes between the server and the browser. Finally, we'll extend the application by changing the popup highlighted symbols.

What is REST? REST stands for Representational State Transfer. It's a software architecture that focuses on the interface between server and client using a hypermedia environment. It limits the actions that can be performed between the client and the server, but provides enough user information and documentation for the user to navigate amongst data and states.

In our discussion of ArcGIS Server and REST in this article, we'll cover the following topics:

Handling REST endpoints and data formats

The hierarchy of ESRI REST Services, as seen through a browser

REST is a methodology based around web pages. A website presents a state (the URL), data transfer (a HTML page, CSS, and JavaScript), and a well-documented way to navigate between the different states (links). While understanding a website to be RESTful is well and good, what makes ArcGIS Server so RESTful?

In order for a web service to be considered RESTful, it must meet the following requirements:

Client-Server: The roles of the client and server are clearly defined. The client doesn't care if the server contains one or one million records, and the server does not depend on a particular UI for the client. As long as the interface between the client and server remains the same, the client and server code can be changed independently.

Stateless: The client handles the state of the application, whereas the server does not have to keep up with it. The client has all it needs to make a request, including the necessary parameters and security tokens.

Cacheable: Sometimes client applications cache data for faster performance because the World Wide Web delivers data asynchronously. The server needs to tell the client which requests can be cached and for how long.

Layered system: The server-side application can be placed behind a load balancer, a secure system, or a proxy, with no noticeable effect on the client-side application.

Code on demand (optional): The server can provide code for the client to run. Examples include Java applets or JavaScript scripts. Not all REST services do this.

Uniform interface: With a REST service, the server provides a uniform interface through which the client can interact with the data. The Uniform Interface can be broken down further into four principals.
- Information requests that include the identification of the resource. This includes everything from the data source to the output type.
- The client has enough information from a request to manipulate or delete data.
- Messages from the server contain instructions about how to use them.
- A state is handled by the client using hypermedia (web pages, query parameters, and session states).

If you look into the ArcGIS Server implementation, you'll see that it meets these criteria. Therefore, it's considered RESTful.

Looking at a map server

ArcGIS Server provides web access to its map service contents. To access the content, you need to know the ArcGIS Server name and the site name. By default, ArcGIS Server is reached through port 6080. It can also be reached through port 80 if it has been configured and licensed to deliver web content. REST Service endpoints can be reached through your browser with this address:

http://<GIS Server Name>:6080/<site name>/rest/services

Where GIS Server Name refers to the ArcGIS Server machine, and site name refers to the ArcGIS Server instance which, by default, is arcgis. The port number is optional if ArcGIS Server has been set up to deliver traffic on port 80, which is the default port for Internet traffic.

When there are multiple GIS Servers, often handling a large load of traffic or complicated services, a web adaptor may be installed. The web adaptor routes traffic to multiple ArcGIS Servers, based on service requests, load balancing, and other related issues. The web adaptor also provides a layer of security, whereby ArcGIS Server machine names are not directly exposed to the outside world. To access the REST service through the web adaptor, use the following URL.

http://<web server name>/<web adaptor name>/rest/services

As long as ArcGIS Server is accessible from our computer, we can access information in the web browser. By default, the service data is presented as HTML. From there we can see the properties of the REST service, and follow links to other services and actions on the server. This lets developers review and test map services, independent of any application they create.

ArcGIS REST services provide a great way to view and interact with service data using HTML, which is good for presentation. Almost all of our applications will interact with the REST service through server requests. ArcGIS Server can therefore communicate through REST using another data format called JSON.

Working with JSON

JavaScript Object Notation (JSON) provides a structured data format for loosely defined data structures. A JSON object is built with other JSON objects, including strings, numbers, other objects, and arrays. Any data is allowed, as long as everything is self-contained and there is no sloppy formatting with missing brackets and braces.

There are a number of ways to test for valid JSON. Visit http://jsonlint.com, where you can copy and paste your JSON and submit it for validation. It will point out missing or broken formatting issues, as well as how to resolve them.

JSON validators require that all string items are enclosed in quotes. Single or double quotes will work, as long as you put the same marks at the end of a string as those at the beginning. This includes both JSON object key fields. JavaScript interpreters in a browser are more flexible, key fields do not have to be enclosed by quotes. It all depends on how you're testing the JSON.

Before JSON was developed, data was passed from server to client in a format called Extensible Markup Language (XML). XML is a document markup language that shows data in a format both humans and machines can read. The XML format can be read and parsed by a number of programming languages.

There are two main reasons why JSON is the preferred data format for web applications when compared to XML. First, JSON data can be consumed immediately by JavaScript applications. XML requires extra steps to parse the data into a usable object. Secondly, JSON data takes up less space. Let's explore that by comparing the following snippets of data. The following snippet is written in XML:

<mountain>
  <name>Mount Everest</name>
  <elevation>29029</elevation>
  <elevationUnit>ft</elevationUnit>
  <mountainRange>Himalaya</mountainRange>
  <dateOfFirstAscent>May 29, 1953</dateOfFirstAscent>
  <ascendedBy>
    <person>
      <firstName>Tenzing</firstName>
      <lastName>Norgay</lastName>
    </person>
    <person>
      <firstName>Edmund</firstName>
      <lastName>Hillary</lastName>
    </person>
  </ascendedBy>
</mountain>

Now, here's the same data written in JSON:

{
  "type": "mountain",
  "name": "Mount Everest",
  "elevation": 29029,
  "elevationUnit": "ft",
  "mountainRange": "Himilaya",
  "dateOfFirstAscent": "May 29, 1953",
  "ascendedBy": [
    {
      "type": "person",
      "firstName": "Tenzing",
      "lastName": "Norgay"
    },
    {
      "type": "person",
      "firstName": "Edmund",
      "lastName": "Hillary"
    }
  ]
}

The same data in JSON counts 62 characters less than that in XML. If we take out the line breaks and extra spaces, or minimize the data, the JSON data is 93 characters shorter than the minimized XML. With bandwidth at a premium, especially for mobile browsers, you can see why JSON is the preferred format for data transmission.

JSON and PJSON formatting

JSON comes in two flavors. The default JSON is minimized, with all the extra spaces and line returns removed. Pretty JSON, or PJSON for short, contains line breaks and spacing to show the structure and hierarchy of the data. The previous Mount Everest example shows what PJSON looks like. While PJSON is easier to read, and therefore easier to troubleshoot for errors, the minimized JSON is much smaller. In the example, the PJSON has 397 characters, while the minimized version has only 277 characters, a 30 percent decrease in size.

When viewing ArcGIS REST service data, you can change the format of the data by adding an f query parameter to the REST Service URL. It should look like the following URL:

http://<GIS web service>/arcgis/rest/services/?f=<format>

Here, you can set f=JSON to receive the raw JSON data, or f=PJSON to receive the human-readable pretty JSON (or padded JSON, if you prefer). Some browsers, such as Google Chrome and Mozilla Firefox, offer third party extensions that reformat raw JSON data into PJSON without making the request.

Service Level

Let's start by viewing the sample ArcGIS Server services at http://sampleserver6.arcgisonline.com/arcgis/rest/services. When we request the page as HTML, we notice a few things. First, the version of ArcGIS Server is shown (version 10.21 at the time of writing). The version number is important because many features and bits of information may not be present in older versions. Secondly, we see a list of links pointing to folders. These are map services grouped in any way the publisher chooses. We also see a list of map service links below the folder lists. Finally, at the bottom of the page, we see supported interfaces. In this site, we can see the REST interface that we're familiar with. Here's a picture of the service:

finding-peace-rest-img-0

If we change the format of the REST Service request in our browser to Pretty JSON, by adding ?f=pjson to the end of the URL, we can see roughly how the ArcGIS JavaScript API would see this location:

finding-peace-rest-img-1

Here, the JSON object returned includes the numeric currentVersion, an array of folder names, and an array of services objects. The service JSON objects contain a name and a type attribute, which tells you what kind of service you're dealing with, and gives you the components you need to construct the URL link to those services. This format is as follows:

http://<server>/arcgis/rest/services/<service.name>/<service.type>

If we follow the link to our census map service, we can see more details.

Map services

A map service gives applications access to map data published with ArcGIS Server. It contains information about the map layout, format, contents, and other items necessary to properly render the map with the various ArcGIS API's. The map service URL is formatted as follows:

http://<ArcGIS Server REST Services>/<mapName>/MapServer

or

http://<ArcGIS Server REST Services>/<folder>/<mapName>/MapServer

When you navigate to a map service using your browser, you're presented with a lot of information about the map service. The HTML provides links to view the data in different applications, including the ArcGIS JavaScript API and ArcMap. Google Earth is also available if the map service is published to serve data in that format. The HTML for the map service also provides a lot of metadata to help you understand what it's offering. These properties include the Description, Service Description, Copyright Text, and the Document Info.

Some of the map service properties can be difficult to understand without some context. We'll review some of the important ones. Remember that properties in this list show how they are listed in the HTML. When shown in JSON, these items are camel-cased (first letter lowercase, no spaces, and capital letters to start each new word after the first).

Spatial reference: How the layout of the map compares with the real world, which we'll discuss a little later.

Single fused map cache: Lets you know whether the map data has been cached, or if it is dynamic. You can load the layer by using either ArcGISTiledMapServiceLayer or ArcGISDynamicMapServiceLayer, respectively.

Initial extent/Full extent: When you first load the map with the ArcGIS JavaScript API, the Initial Extent describes the bounding box of the area you see the first time. The Full Extent is the expected full area of the map service, which may be much wider than all the data.

Supported image format types: When ArcGIS Server draws the map layers as tiles, these are the image formats that can be returned. PNG32 is recommended if your data has a lot of semi-transparencies and colors, while PNG8 works well with very simple symbols.

Supports dynamic layers: If true, the developer can change the symbology and layer definitions when displaying the map service.

Max record count: When submitting a query, identify or some other search, this is the maximum number of results that can be returned by the map service. This information can only be changed by server-side changes to the map service.

Finally, the Map Service HTML provides links to a number of related REST Service endpoints. Most of these links extend the existing URL and provide more information about the map service. As a bare minimum, the following should be present:

Legend: Displays the symbology of the layers in the map service.

Export map: This feature lets you download an image showing an area of the map that fits within a specific bounding box. You can specify parameters.

Identify: This lets you identify features within all layers of a map service, based on the geometry passed in. This functionality is used by IdentifyTask.

Find: This lets the user search for features based on the presence of a line of text passed to it. This functionality is implemented by FindTask.

Map service layers

When exploring the layers of a map service, it helps to know what to look for. Map services list the basic contents of their layers within an array of objects in their layers property.

All layer objects have the same format, with the same properties. Each layer object has a numeric id property that refers to the layer's zero-based position in the list. Layer objects also have a name property that comes from how the layer was named in the map service. These layers also have a minScale and maxScale property, showing the range within which the layer is visible (with a 0 value meaning there is no minScale or maxScale limitation). When determining visibility, the layer object also contains a Boolean defaultVisibility property that describes whether the layer is initially visible when the map service loads.

Map service layer objects also contain information about their layer hierarchy. Each map layer object contains a parentLayerId and a subLayerIds property. The parentLayerId is a number that refers to the index of the parent group layer for the specific layer. A parent layer id of -1 means the layer in question has no parent layer. The subLayerIds are an integer array of the indexes where you can find the sublayers for the particular parent layer. If a layer has no sub layers, the subLayerIds will be a null value, instead of an empty list. You can see an example of map service layers in the following code:

layers: [
  {
    "id" : 0, 
    "name" : "Pet Lovers", 
    "parentLayerId" : -1, 
    "defaultVisibility" : true, 
    "subLayerIds" : [1, 2], 
    "minScale" : 16000
    "maxScale" : 0
  },
  {
    "id" : 1, 
    "name" : "Dog Lovers", 
    "parentLayerId" : 0, 
    "defaultVisibility" : true, 
    "subLayerIds" : null, 
    "minScale" : 16000
    "maxScale" : 0
  },
  {
    "id" : 2, 
    "name" : "Cat Lovers", 
    "parentLayerId" : 0, 
    "defaultVisibility" : true, 
    "subLayerIds" : null, 
    "minScale" : 16000
    "maxScale" : 0
  }
],

In the preceding snippet, the map service has three layers. The Pet Lovers layer is actually a parentLayer, and corresponds to a group layer assigned in an ArcMap .mxd file. There are two layers in parentLayer, Dog Lovers and Cat Lovers. All layers are visible by default, and the layers do not appear until the map is at a scale lower than 1:16,000, according to minScale. The maxScale property is set to zero, meaning there is no maximum scale where the layer turns off again.

Feature services

Feature services are similar to map services, but provide more functionality. Their content can be edited, if the database and map settings support those operations. They display their feature symbology without the need of a legend service. Their symbology can also be modified client-side, by changing their renderer. The URL of a feature service is similar to a map service, except that it ends with FeatureServer, as shown in the following:

http://<GIS-web-server>/arcgis/rest/services/<folder>/<mapname>/FeatureServer

The feature service differs first and foremost in its capabilities. Apart from allowing you to query data, feature service capabilities allow the user to create, update, and/or delete records. Those familiar with CRUD operations will recognize those words as the C, U, and D in CRUD (the R stands for read, which is what happens when you query for results). The capabilities include editing if create, update, or delete are allowed. Also, if the feature service supports file attachments to data, such as photos, the capabilities will include the word "upload".

There are other feature service properties that may help you learn more about the service. They include the following:

Has Versioned Data: Lets you know that the geodatabase has versioning enabled, which allows edits to be undone/redone.

Supports Disconnected Editing: Data can be checked out and edited in an environment without an Internet connection. When the application connects to the Internet again, the data can be checked back in.

Sync Enabled: If this is true, feature data can be synced between the geodatabase the data comes from, and another geodatabase.

Allow Geometry Updates: If editing is allowed, this lets the API know if the feature geometries can be edited or not. Due to certain permissions, the application might only allow for updates to the feature attributes, while the geometries remain unchanged.

Enable Z Defaults: If the data contains height data (z), default values are assigned in the map service.

Layer level

Map services and feature services are made up of layers. These layers group together geographic features with the same geometry type and the same sets of properties. Layers are referred to by their numerical index in the list. The layer index starts at 0 for the bottom layer, and goes up one for each additional layer. The URL might look something like this for the first layer in a map service:

http://<GIS-web-server>/arcgis/rest/services/<folder>/<mapname>/MapServer/0

Map layers offer a whole host of data to help you understand what you're viewing. The layer's name property comes either from its name in the .mxd file, or from the layer in the Table of Contents, if the file is unsaved. The map layer also provides a description, and copyright data. The Display Field property tells the map service what to use when labeling features, if labeling is turned on.

Map layers also provide important data that you can use in your application. The type parameter tells you the geometry of the layer, whether it's a point, line, or polygon. Default Visibility lets you know if the layer was originally visible or not when the map service began. Min Scale and Max Scale affect visibility, depending on your zoom level. The map service also lets you know if the layers have attachments, can be modified with different renderers, and how many results can be returned from a query.

Fields

A map service layer provides information about its attribute by means of the field property. The field property is an array of field objects with similar formats. All fields have a type, a name, and an alias attribute. The type refers to the data type of the field, whether it's a string or an integer, or something else. A current list of supported types can be found at http://resources.arcgis.com/en/help/arcgis-rest-api/#/field/02r300000051000000/. The name attribute is the field name for the property, as found in the geodatabase. Field names don't contain spaces or special characters.

The alias field is a string that shows the field name for presentation purposes. Unlike the field name, the alias can have spaces or other special characters. If no alias is assigned in the geodatabase or the map service, the alias field is the same as the field name. For instance, when creating the map service with ArcMap, you might have some data for a block with a field name NUMB_HSES. If you want to show the values for this property in a chart, the field name may look rough and a little confusing. You can then add an alias to the NUMB_HSES field by calling it Number of Houses. That alias provides a much better description for the field:

{
  "type": "esriFieldTypeInteger",
  "name" "NUMB_HSES",
  "alias": "Number of Houses"
}

Domains

Field objects may also have domain attributes assigned to them. Domains are limitations on field values imposed at the geodatabase level. Domains are uniquely created in the geodatabase, and can be assigned to feature classes and table fields. Domains make it easier to input the correct values by restricting what can be entered. Instead of allowing users to mistype street names in a report service, for instance, you might provide a field with a domain containing all the correctly typed street names. The user can then select from the list, rather than have to guess how to spell the street name.

The ArcGIS REST API supports two varieties of domains: ranges and coded values. Ranges, as the name implies, set a minimum and maximum numeric value for a feature attribute. One example of a range might be an average user rating for restaurants. The restaurant might get somewhere between one and five stars, so you wouldn't want a restaurant to accidently get a value of 6 or a value of less than 1. You can see an example of a rating field with that range domain in this snippet:

{
  "type": "esriFieldTypeInteger",
  "name": "RATING",
  "alias": "Rating",
  "domain": {
    "type": "range",
    "name": "Star Rating",
    "range": [1, 5]
  }
}

A coded value domain provides a list of code and value pairs to use as legitimate property values. The coded value list contains items with a name and a code. The code is the value stored in the geodatabase. The name is the text representation of the coded value. They're useful in that users are forced to select a valid value, instead of mistyping a correct value.

In the following example, we can see a field with a coded value domain. The field contains state abbreviations, but the domain allows the user to see entire state names:

{
  "type": "esriFieldTypeString",
  "name": "STATE",
  "alias": "State",
  "length": 2,
  "domain": {
    "type": "codedValue",
    "name": "State Abbreviation Codes",
    "codedValues": [
      {"name": "Alabama", "code": "AL"},
      {"name": "Alaska", "code": "AK"},
      {"name": "Wisconsin", "code": "WI"},
      {"name": "Wyoming", "code": "WY"}
    ]
  }
}

In the preceding example, state names are stored in two letter code form. The domain provides a full name reference table with the full names of the states. If you were to send queries for features using this field, you would use the code values. Querying for all features where STATE = 'Alaska' would yield no results, while a query where STATE = 'AK' may give you results.

Note that the code and the value don't have to be of the same type. You can have numeric codes for, say, water line part numbers, and coded values to show their descriptive names.

Related tables

Tables with non-geographic data can be published in a map service. These tables may provide data related to map features, such as comments on campground locations or the sales history of a property. These tables can be searched and queried like features. Relationships between map layers and tables can also be published and searched.

Layers and tables can be joined, either by using geodatabase relationship classes, or ad-hoc relationship assignments in ArcMap. When published with ArcMap, those relationships are preserved in ArcGIS Server. The connection between related features and tables is stored within the relationships property of the layer and table. A developer can query related data, based on a selection in the parent feature class.

Relationship objects have the same general format. Each relationship object contains a numerical id and relatedTableId. The relatedTableId is linked to the RelationshipQuery object to query for related results. The role describes whether the current layer or table is the origin or the destination of the relationship. Cardinality describes whether a single origin object has one or several destinations related to it.

When querying for results, results return much faster if you start with the origin and use RelationshipQuery on the destination tables. Starting with the destination tables may take significantly longer.

Summary

We have explored the different parts of the REST Service endpoints for ArcGIS Server, the primary data source for our ArcGIS JavaScript API-based applications. We've learned what it means for a service to be RESTful, and how that applies to ArcGIS Server. We've explored the organization of ArcGIS Server map services.

With your understanding of how data is handled client-side and server-side, you will be able to implement many of the powerful features that ArcGIS Server offers for web-based applications.