GIS in XML

Fig 1 – DataConnector

Web mapping has generally been a 3 tier proposition. In a typical small to medium scale scenario there will almost always be a data source residing in a spatial database, with some type of service in the middle relaying queries from a browser client UI to the database and back.

I’ve worked with all kinds of permutations of 3 tier web mapping, some easier to use than others. However, a few months ago I sat down with all the Microsoft offerings and worked out an example using SQL Server + WCF + Bing Maps Silverlight Control. Notice tools for all three tiers are available from the same vendor i.e. Microsoft Visual Studio. I have to admit that it is really nice to have integrated tools across the whole span. The Microsoft option has only been possible in the last year or so with the introduction of SQL Server 2008 and Bing Maps Silverlight Map control.

The resulting project is available on codeplex: dataconnector
and you can play with a version online here: DataConnectorUI

When working on the project, I was starting out in WCF with some reservations. SQL Spatial was similar to much of my earlier work with PostGIS. While Bing Maps Silverlight Control and XAML echoed work I’d done a decade back with SVG, just more so. However, for the middle tier I had generally used something from the OGC world such as GeoServer. Putting together my own middle tier service using WCF was largely experimental. WCF turned out to be less daunting than I had anticipated. In addition, it also afforded opportunity to try out a few different approaches for transferring spatial query results to the UI client.

There is more complete information on the project here:
http://dataconnector.codeplex.com/documentation

After all the experimental approaches my conclusion is that even with the powerful CLR performance of Silverlight Bing Maps Control, most scenarios still call for a hybrid approach: raster performance tile pyramids for large extent or dense data resources, and vectors for better user interaction at lower levels in the pyramid.

Tile Pyramids don’t have to be static and DataConnector has examples of both static and dynamic tile sources. The static example is a little different from other approaches I’ve used, such as GeoWebCache, since it drops tiles into SQL Server as they are created rather than using a file system pyramid. I imagine that a straight static file system source could be a bit faster, but it is nice to have indexed quadkey access and all data residing in a single repository. This was actually a better choice when deploying to Azure, since I didn’t have to work out a blob storage option for the tiles in addition to the already available SQL Azure.

Hybrid web mapping:

Here are some of the tradeoffs involved between vector and raster tiles.

Fig 2 – Raster Tiles vs Vector

Hybrid architectures switch between the two depending on the client’s position in the resource pyramid. Here is my analysis of feature count ranges and optimal architecture:

1. Low – For vector poly feature counts < 300 features per viewport, I’d use vector queries from DB. Taking advantage of the SQL Reduce function makes it possible to drop node counts for polygons and polylines for lower zoom levels. Points are more efficient and up to 3000-5000 points per viewport are still possible.

2. Medium – For zoomlevels with counts > 300 per viewport, I’d use a dynamic tile builder at the top of the pyramid. I’m not sure what the upper limit is on performance here. I’ve only run it on fairly small tables 2000 records. Eventually dynamic tile building on the server effects performance (at the server not the client).

3. High – For zoomlevels with high feature counts and large poly node counts, I’d start with pre-seeded static tile at low zoomlevels at the top of the pyramid, perhaps dynamic tiles in the middle pyramid, and vectors at the bottom.

4. Very High – For very high feature counts at low zoomlevels near the top of the pyramid I’d just turn off the layer. There probably isn’t much reason to show very dense resources until the user moves in to an area of interest. For dense point sources a heat map raster overview would be best at the top of the pyramid. At middle levels I’d use a caching tile builder with vectors again at higher zoom levels at the bottom of the pyramid.

Here is a graphic view of some hybrid architectural options:

Data Distribution Geographically:
Another consideration is the distribution of data in the resource. Homogenous geographic data density works best with hard zoom level switches. In other words, the switch from vector to raster tile can be coded to zoomlevel regardless of where the client has panned in the extent of the data. This is simple to implement.

However where data is relatively heterogeneous geographically it might be nice to arrange switching according to density. An example might be parcel data densities that vary across urban and rural areas. Instead of simple zoom levels, the switch between tile and vector is based on density calculations. Having available a heat map overview, for example, could provide a quick viewport density calculation based on a pixel sum of the heat map intersecting with the user’s viewport. This density calculation would be used for the switch rather than a simpler zoom level switch. This way rural area of interest can gain the benefit of vectors higher in the pyramid than would be useful in urban areas.

Point Layers:
Points have a slightly different twist. For one thing too many points clutter a map, while their simplicity means that more point vectors can be rendered before affecting UI performance. Heat Maps are a great way to show density at higher levels in the pyramid. Heat Maps can be dynamic tile source or a more generalized caching tile pyramid. In a point layer scenario at some level there is a switch from Heat Map to Cluster Icons, and then to individual Pushpins. Using power scaling at pushpin levels allows higher density pins to show higher in the pyramid without as much clutter. Power scaling hooks the icon size for the pin to zoomlevel. Experiments showed icon max limit for Bing Silverlight Map Control at 3000-5000 per viewport.

Some Caveats:
Tile pyramids are of course most efficient when the data is relatively static. With highly dynamic data, tiles can be built on the fly but with consequent loss of performance as well as loading on the server that affects scaling. In an intermediate situation with data that changes slowly, static tiles are still an option using a pre-seeding batch process run at some scheduled interval.

Batch tile loading also has limitations for very dense resources that require tiling down deep in a pyramid where the number of tiles grows very large. Seeding all levels of a deep pyramid requires some time, perhaps too much time. However, in a hybrid case this should rarely happen since the bottom levels of the pyramid are handled dynamically as vectors.

It is also worth noting that with Silverlight the client hardware affects performance. Silverlight has an advantage for web distribution. It distributes the cpu load out to the clients harnessing all their resources. However, an ancient client with old hardware performance will not match performance of newer machines.

Conclusion:

A Hybrid zoom level based approach is the best general architecture for Silverlight web maps where large data sets are involved. Using your own service provides more options in architecting a web map solution.

Microsoft’s WCF is a nice framework especially since it leverages the same C# language, IDE, and debugging capabilities across all tiers of a solution. Microsoft is the only vendor out there with the breadth to give developers an integrated solution across the whole spectrum from UI graphics, to customizable services, to SQL spatial tables.

Then throw in global map and imagery resources from Bing, with Geocode, Routing, and Search services, along with the Azure cloud platform for scalability and it’s no wonder Microsoft is a hit with the enterprise. Microsoft’s full range single vendor package is obviously a powerful incentive in the enterprise world. Although relatively new to the game, Microsoft’s full court offense puts some pressure on traditional GIS vendors, especially in the web distribution side of the equation.

Fig 1 – Showing a sequence of LiDAR Profiles as a video clip

Video codecs harnessed to Silverlight MediaPlayer make a useful compression technique for large image sets. The video, in essence, is a pointer into a large library of image frames. Data collected in a framewise spatial sense can leverage this technique. One example where this can be useful is in Mobile Asset Collection or MAC UIs. MAC corridors can create large collections of image assets in short order.

Another potential for this approach is to make use of conventional LiDAR point cloud profiles. In this project I wanted to step through a route, collecting profiles for later viewing. The video approach seemed feasible after watching cached profiles spin through a view panel connected to a MouseWheel event in another project. With a little bit of effort I was able to take a set of stepwise profiles, turn them into a video, and then connect the resulting video to a Silverlight Map Control client hooked up to a Silverlight MediaPlayer. This involved three steps:

1. Create a set of frames

Here is a code snippet used to follow a simple track down Colfax Ave in Denver, west to east. I used this to repeatedly grab LidarServer WMS GetProfileData requests, and save the resulting .png images to a subdirectory. The parameters were set to sweep at a 250ft offset either side of the track with a10ft depth and a 1 ft step interval. The result after a couple of hours was 19164 .png profiles at 300px X 500px.

The code basically starts down the supplied path and calculates the sweep profile endpoints at each step using the Perpendicular function. This sweep line supplies the extent parameters for a WMS GetProfileData request.

private void CreateImages(object sender, RoutedEventArgs e)
{
    string colorization = "Elevation";
    string filter = "All";
    string graticule = "True";
    string drapeline = "False";
    double profileWidth = 300;
    double profileHeight = 500;

    double dx = 0;
    double dy = 0;
    double len = 0;
    double t = 0;
    string profileUrl = null;
    WebClient client = new WebClient();

    step = 57.2957795 * (step * 0.3048 / 6378137.0); //approx dec degree
    Point startpt = new Point();
    Point endpt = new Point();
    string[] lines = points.Text.Split('\r');
    startpt.X = double.Parse(lines[0].Split(',')[0]);
    startpt.Y = double.Parse(lines[0].Split(',')[1]);

    endpt.X = double.Parse(lines[1].Split(',')[0]);
    endpt.Y = double.Parse(lines[1].Split(',')[1]);

    dx = endpt.X - startpt.X;
    dy = endpt.Y - startpt.Y;
    len = Math.Sqrt(dx * dx + dy * dy);

    Line direction = new Line();
    direction.X1 = startpt.X;
    direction.Y1 = startpt.Y;
    width *= 0.3048;
    int cnt = 0;
    t = step / len;

    while (t <= 1)
    {
        direction.X2 = startpt.X + dx * t;
        direction.Y2 = startpt.Y + dy * t;

        Point p0 = Perpendicular(direction, width / 2);
        Point p1 = Perpendicular(direction, -width / 2);

        p0 = Mercator(p0.X, p0.Y);
        p1 = Mercator(p1.X, p1.Y);

        profileUrl = "http://www.lidarserver.com/drcog?SERVICE=WMS&VERSION=1.3&REQUEST=GetProfileView"+
                          "&FORMAT=image%2Fpng&EXCEPTIONS=INIMAGE&CRS=EPSG:3785"+
                         "&LEFT_XY=" + p0.X + "%2C" + p0.Y + "&RIGHT_XY=" + p1.X + "%2C" + p1.Y +
                         "&DEPTH=" + depth + "&SHOW_DRAPELINE=" + drapeline + "&SHOW_GRATICULE=" + graticule +
                         "&COLORIZATION=" + colorization + "&FILTER=" + filter +
                         "&WIDTH=" + profileWidth + "&HEIGHT=" + profileHeight;

        byte[] bytes = client.DownloadData(new Uri(profileUrl));
        FileStream fs = File.Create(String.Format(workingDir+"img{0:00000}.png", cnt++));
        BinaryWriter bw = new BinaryWriter(fs);
        bw.Write(bytes);
        bw.Close();
        fs.Close();

        direction.X1 = direction.X2;
        direction.Y1 = direction.Y2;
        t += step / len;
    }
}

private Point Perpendicular(Line ctrline, double dist)
{
    Point pt = new Point();
    Point p1 = Mercator(ctrline.X1, ctrline.Y1);
    Point p2 = Mercator(ctrline.X2, ctrline.Y2);

    double dx = p2.X - p1.X;
    double dy = p2.Y - p1.Y;
    double len = Math.Sqrt(dx * dx + dy * dy);
    double e = dist * (dx / len);
    double f = dist * (dy / len);

    pt.X = p1.X - f;
    pt.Y = p1.Y + e;
    pt = InverseMercator(pt.X, pt.Y);
    return pt;
}

2. Merge the .png frames into an AVI

Here is a helpful C# AviFile library wrapper. Even though it is a little old, the functions I wanted in this wrapper worked just fine. The following WPF project simply takes a set of png files and adds them one at a time to an avi clip. Since I chose the (Full)uncompressed option, I had to break my files into smaller sets to keep from running into the 4Gb limit on my 32bit system. In the end I had 7 avi clips to cover the 19,164 png frames.

Fig 2 - Create an AVI clip from png frames

using System.Drawing;
using System.Windows;
using AviFile;

namespace CreateAVI
{

    public partial class Window1 : Window
    {
        public Window1()
        {
            InitializeComponent();
        }

        private void btnWrite_Click(object sender, RoutedEventArgs e)
        {
            int startframe = int.Parse(start.Text);
            int frameInterval = int.Parse(interval.Text);
            double frameRate = double.Parse(fps.Text);
            int endframe = 0;

            string currentDirName = inputDir.Text;
            string[] files = System.IO.Directory.GetFiles(currentDirName, "*.png");
            if (files.Length > (startframe + frameInterval)) endframe = startframe + frameInterval;
            else endframe = files.Length;
            Bitmap bmp = (Bitmap)System.Drawing.Image.FromFile(files[startframe]);
            AviManager aviManager = new AviManager(@currentDirName + outputFile.Text, false);
            VideoStream aviStream = aviManager.AddVideoStream(true, frameRate, bmp);

            Bitmap bitmap;
            int count = 0;
            for (int n = startframe+1; n < endframe; n++)
            {
                if (files[n].Trim().Length > 0)
                {
                    bitmap = (Bitmap)Bitmap.FromFile(files[n]);
                    aviStream.AddFrame(bitmap);
                    bitmap.Dispose();
                    count++;
                }
            }
            aviManager.Close();
        }
    }
}

Next I used Microsoft Expression Encoder 3 to encode the set of avi files into a Silverlight optimized VC-1 Broadband variable bitrate wmv output, which expects a broadband connection for an average 1632 Kbps download. The whole path sweep takes about 12.5 minutes to view and 53.5Mb of disk space. I used a 25fps frame rate when building the avi files. Since the sweep step is 1ft this works out to about a 17mph speed down my route.

3. Add the wmv in a MediaPlayer and connect to Silverlight Map Control.

MediaPlayer

I used a similar approach for connecting a route path to a video described in "Azure Video and the Silverlight Path". Expression Encoder 3 comes with a set of Silverlight MediaPlayers templates. I used the simple "SL3Standard" template in this case, but you can get fancier if you want.

Looking in the Expression Templates subdirectory "C:\Program Files\Microsoft Expression\Encoder 3\Templates\en", select the ExpressionMediaPlayer.MediaPlayer template you would like to use. All of the templates start with a generic.xaml MediaPlayer template. Add the .\Source\MediaPlayer\Themes\generic.xaml to your project. Then look through this xaml for <Style TargetType="local:MediaPlayer">. Once a key name is added, this plain generic style can be referenced by MediaPlayer in the MainPage.xaml
<Style x:Key="MediaPlayerStyle" TargetType="local:MediaPlayer">

It is a bit more involved to add one of the more fancy templates. It requires creating another ResourceDictionary xaml file, adding the styling from the template Page.xaml and then adding both the generic and the new template as merged dictionaries:

Removing unnecessary controls, like volume controls, mute button, and misc controls, involves finding the control in the ResourceDictionary xaml and changing Visibility to Collapsed.

Loading Route to Map

The list of node points at each GetProfileData frame was captured in a text file in the first step. This file is added as an embedded resource that can be loaded at initialization. Since there are 19164 nodes, the MapPolyline is reduced using only modulus 25 nodes resulting in a more manageable 766 node MapPolyline. The full node list is still kept in a routeLocations Collection. Having the full node list available helps to synch with the video timer. This video is encoded at 25fps so I can relate any video time to a node index.

private List routeLocations = new List();

 private void LoadRoute()
 {
     MapPolyline rte = new MapPolyline();
     rte.Name = "route";
     rte.Stroke = new SolidColorBrush(Colors.Blue);
     rte.StrokeThickness = 10;
     rte.Opacity = 0.5;
     rte.Locations = new LocationCollection();

     Stream strm = Assembly.GetExecutingAssembly().GetManifestResourceStream("OnTerra_MACCorridor.corridor.direction.txt");

     string line;
     int cnt = 0;
     using (StreamReader reader = new StreamReader(strm))
     {
         while ((line = reader.ReadLine()) != null)
         {
             string[] values = line.Split(',');
             Location loc = new Location(double.Parse(values[0]), double.Parse(values[1]));
             routeLocations.Add(loc);
             if ((cnt++) % 25 == 0) rte.Locations.Add(loc);// add node every second
         }
     }
     profileLayer.Children.Add(rte);
 }

A Sweep MapPolyline is also added to the map with an event handler for MouseLeftButtonDown. The corresponding MouseMove and MouseLeftButtonUp events are added to the Map Control, which sets up a user drag capability. Every MouseMove event calls a FindNearestPoint(LL, routeLocations) function which returns a Location and updates the current routeIndex. This way the user sweep drag movements are locked to the route and the index is available to get the node point at the closest frame. This routeIndex is used to update the sweep profile end points to the new locations.

Synchronzing MediaPlayer and Sweep location

From the video perspective a DispatcherTimer polls the video MediaPlayer time position every 500ms. The MediaPlayer time position returned in seconds is multiplied by the frame rate of 25fps giving the routeLocations node index, which is used to update the sweep MapPolyline locations.

In reverse, a user drags and drops the sweep line at some point along the route. The MouseMove keeps the current routeIndex updated so that the mouse up event can change the sweep locations to its new location on the route. Also in this MouseLeftButtonUp event handler the video position is updated dividing the routeIndex by the frame rate.
VideoFile.Position = routeIndex/frameRate;

Summary

Since Silverlight handles media as well as maps it's possible to take advantage of video codecs as a sort of compression technique. In this example, all of the large number of frame images collected from a LiDAR point cloud are readily available in a web mapping interface. Connecting the video timer with a node collection makes it relatively easy to keep map position and video synchronized. The video is a pointer into the large library of LiDAR profile image frames.

From a mapping perspective, this can be thought of as a raster organizational pattern, similar in a sense to tile pyramids. In this case, however, a single time axis is the pointer into a large image set, while with tile pyramids 3 axis pointers access the image library with zoom, x, and y. In either case visually interacting with a large image library enhances the human interface. My previous unsuccessful experiment with video pyramids attempted to combine a serial time axis with the three tile pyramid axis.I still believe this will be a reality sometime.

Of course there are other universes than our earth's surface. It seems reasonable to think dynamic visualizer approaches could be extended to other large domains. Perhaps Bioinformatics could make use of tile pyramids and video codecs to explore Genomic or Proteomic topologies. It would be an interesting investigation.

Bioinformatics is a whole other world. While we are playing around in “mirror land” these guys are doing the “mirror us.”

Fig 1 - MediaPlayer using BlackGlass template