Approach for Push-Based Result Transmission with Asynchronous Geospatial Processing

Geospatial analyses are increasingly being conducted in networked environments. In such cases, geographic analyses are carried out by leveraging ressources (including geospatial algorithms) on some remote server. A user might upload some dataset onto a server, triggers some geo-computation and awaits the results.

Most of such web-based geo-processing services are implementing the so called “Web Processing Service” (WPS) standard, which is provided by the OGC. This is an industry standard for serving access to geospatial processing capabilities. It allows performing analyses in two different ways, either in a synchronous mode or asynchronoulsy. The synchronous mode finishes all action within one request-response cycle. In such case, there is no need for further action at the user. The results are instantaneously responded once the processing is finished. However, it is obvious that this mode is not suitable for highly complex processing tasks. These can take up to several hours, almost certainly exceeding the common server time-out ranges (typically around 30s). This is where the second option, asynchronous processing, comes into play. In that case, the user triggers the desired computation and receives an URL. This URL references the location where the results will be stored upon completion. However, the user is not aware when this will happen. Thus, the user constantly has to check the URL for results, resulting into large network overheads when being performed in an automated way. This is particularly a problem when geographic analyses are included in some automated processing chain.

One way of overcoming this problem is the so called “web socket” technology. This technology is part of the HTML 5 initiative and enables servers for sending data to clients. Older approaches for this purpose have often misused HTTP by artifically keeping the connection open or emulating unfinished data transmission (e.g., like with long polling or iframe streaming). However, HTTP has been designed for unidirectional client-to-server communication. It is by no means suited for two-way communication. Thus, all these approaches are unstandardised, which leads to interoperability problems in case of service-oriented architectures. The web sockets technology overcomes these issues. It switches communication from HTTP to a self-standing protocol. This protocol has specifically been designed for communication in both directions.

We combine WPS and web sockets by extending the asynchronous mode of WPS. Once a client has triggered a computational task, a URL to a web socket server is responded. The client now has to listen to that server. Instead of constantly exchanging HTTP headers, the connection stays idle until it gets invoked by one of the two communication partners. Once the geoprocessing is done, WPS leverages web socket technology for sending a notification about process completion. The client receives this notification, containing a URL referencing the results. This approach enables integration of long-lasting geoprocessing tasks into automated processing chains and saves ressources at the client, the server and in the network. In the corresponding article (see below) we also present suggestions for how to alter the current WPS standard towards push-based communication like described above.

Reference

Westerholt, R. & Resch, B. (2014): Asynchronous Geospatial Processing: An Event-Driven Push-Based Architecture for the OGC Web Processing Service. Transactions in GIS, DOI: 10.1111/tgis.12104.