FlatGeobuf ¶

Author:: Jeff McKenna
Contact:: jmckenna at gatewaygeomatics.com
Last Updated:: 2022-08-12

FlatGeobuf is an optimized binary encoding for vectors based on Flatbuffers, that focuses on read/query performance by leveraging a Packed Hilbert R-Tree spatial index, that enables fast bounding box spatial filtering. FlatGeobuf performance has helped make it become one of the chosen formats for streaming spatial data through HTTP / «the cloud».

More Information ¶

More information about FlatGeobuf is available at:

GitHub repository: https://github.com/flatgeobuf/flatgeobuf
Twitter: https://twitter.com/flatgeobuf

File listing ¶

FlatGeobuf files are usually a single binary file with a FGB filename extension, such as:

countries.fgb

Shënim

Windows users who want to follow along, can convert MS4W’s included demo.db SpatiaLite database into a FlatGeobuf file (for the countries table) with the command (executed inside /ms4w/apps/local-demo/data/) :

ogr2ogr -f FlatGeobuf countries.fgb demo.db countries

Data Access / Connection Method ¶

Here are some rudimentary map draw speed results, with the MapServer 8.0 release code (running on Windows with MS4W):

FlatGeobuf (native) 0.008s
Shapefile (native)  0.010s
FlatGeobuf (OGR)    0.013s
Shapefile (OGR)     0.023s
GeoPackage (OGR)    0.042s
SpatiaLite (OGR)    0.045s
PostGIS (native)    0.053s
GeoJSON (OGR)       0.089s

Method 1: Direct connection (native access)¶

Shihni edhe

MS RFC 137: Native FlatGeobuf support

Native access is available since the MapServer 8.0.0 release, that uses code within the MapServer source to access the FGB file directly, as follows:

CONNECTIONTYPE flatgeobuf
DATA "file.fgb"

Ndihmëz

For MapScript, you can use the setConnectionType()` method, such as

$layer->setConnectionType(MS_FLATGEOBUF, "");

Method 2: Connect through OGR ¶

FlatGeobuf access is also available in MapServer through OGR’s FlatGeobuf driver. You must use GDAL version 3.1.0 or later for FlatGeobuf access.

OGR uses the names of spatial layers within the FlatGeobuf file as layers.

The CONNECTION parameter must include the .fgb extension, and the DATA parameter should be the name of the OGR layer.

CONNECTIONTYPE OGR
CONNECTION "name.fgb"
DATA "layername"

Ndihmëz

Since the MapServer 7.6.0 release, you can also specify OGR driver options, and for the FlatGeobuf driver you can disable checking for invalid data, which is critical for maximum performance

CONNECTIONOPTIONS
  "VERIFY_BUFFERS" "NO" # for maximum performance
END

Step 1: Use ogrinfo to examine ¶

First you should make sure that your local GDAL/OGR build contains the «FlatGeobuf» driver, by using the –formats command:

ogrinfo --formats

   Supported Formats:
        ...
        FlatGeobuf -vector- (rw+v): FlatGeobuf
        ...

Once you have confirmed that you have the FlatGeobuf driver, you are ready to try an ogrinfo command on your file to get a list of spatial layers:

ogrinfo countries.fgb

  INFO: Open of `countries.fgb'
      using driver `FlatGeobuf' successful.
  1: countries (Multi Polygon)

Now use ogrinfo to get information on the structure of the spatial ‹countries› layer:

ogrinfo countries.fgb countries -summary

      INFO: Open of `countries.fgb'
            using driver `FlatGeobuf' successful.

      Layer name: countries
      Geometry: Multi Polygon
      Feature Count: 177
      Extent: (-180.000000, -90.000000) - (180.000000, 83.645130)
      Layer SRS WKT:
      GEOGCRS["WGS 84",
          ENSEMBLE["World Geodetic System 1984 ensemble",
              MEMBER["World Geodetic System 1984 (Transit)"],
              MEMBER["World Geodetic System 1984 (G730)"],
              MEMBER["World Geodetic System 1984 (G873)"],
              MEMBER["World Geodetic System 1984 (G1150)"],
              MEMBER["World Geodetic System 1984 (G1674)"],
              MEMBER["World Geodetic System 1984 (G1762)"],
              MEMBER["World Geodetic System 1984 (G2139)"],
              ELLIPSOID["WGS 84",6378137,298.257223563,
                  LENGTHUNIT["metre",1]],
              ENSEMBLEACCURACY[2.0]],
          PRIMEM["Greenwich",0,
              ANGLEUNIT["degree",0.0174532925199433]],
          CS[ellipsoidal,2],
              AXIS["geodetic latitude (Lat)",north,
                  ORDER[1],
                  ANGLEUNIT["degree",0.0174532925199433]],
              AXIS["geodetic longitude (Lon)",east,
                  ORDER[2],
                  ANGLEUNIT["degree",0.0174532925199433]],
          USAGE[
              SCOPE["Horizontal component of 3D system."],
              AREA["World."],
              BBOX[-90,-180,90,180]],
          ID["EPSG",4326]]
      Data axis to CRS axis mapping: 2,1
      featurecla: String (0.0)
      scalerank: Integer (0.0)
      labelrank: Integer (0.0)
      sovereignt: String (0.0)
      sov_a3: String (0.0)
      adm0_dif: Integer (0.0)
      level: Integer (0.0)
      type: String (0.0)
      ...

Step 2: Add the layer in your mapfile ¶

Method 1: Direct connection (native access)¶

Available since the 8.0.0 release, MapServer can access the FGB file directly, as follows:

/* Countries */
LAYER
  NAME "countries"
  TYPE POLYGON
  STATUS ON
  CONNECTIONTYPE flatgeobuf
  DATA "countries.fgb"
  EXTENT -180.000000 -90.000000 180.000000 83.645130 # for maximum performance
  PROCESSING "CLOSE_CONNECTION=DEFER" # for maximum performance
  CLASS
    NAME "World Countries"
    STYLE
      COLOR 200 200 200
      OUTLINECOLOR 0 0 0
      WIDTH 0.1
    END #style
  END #class
END #layer

Method 2: Connect through OGR ¶

For OGR connections, it is always recommended to set CONNECTIONTYPE, CONNECTION, and DATA, as follows:

/* Countries */
LAYER
  NAME "countries"
  TYPE POLYGON
  STATUS ON
  CONNECTIONTYPE OGR
  CONNECTION "countries.fgb"
  DATA "countries"            # the OGR layername, found through ogrinfo
  CONNECTIONOPTIONS
    "VERIFY_BUFFERS" "NO"     # for maximum performance (requires MapServer >=7.6.0)
  END #connectionoptions
  EXTENT -180.000000 -90.000000 180.000000 83.645130 # for maximum performance
  PROCESSING "CLOSE_CONNECTION=DEFER" # for maximum performance
  CLASS
    NAME "World Countries"
    STYLE
      COLOR 200 200 200
      OUTLINECOLOR 0 0 0
      WIDTH 0.1
    END #style
  END #class
END #layer

Step 3: Test your Mapfile with map2img ¶

Use the MapServer commandline utility map2img to verify that your mapfile creates a valid map image, and also display draw times, such as:

map2img -m flatgeobuf.map -o ttt.png -map_debug 3

msDrawMap(): rendering using outputformat named png (AGG/PNG).
msDrawMap(): WMS/WFS set-up and query, 0.000s
msDrawMap(): Layer 0 (countries), 0.012s
msDrawMap(): Drawing Label Cache, 0.002s
msDrawMap() total time: 0.015s
msSaveImage(flatgeobuf.png) total time: 0.008s

Optional: Configure your WFS service to Output FlatGeoBuf ¶

Shënim

The following is not available through native FlatGeoBuf access.

Since MapServer 6.0, you can configure an OUTPUTFORMAT for your WFS (or WMS) service, for an OGR connection. See the OGR Output document for more information.

First, you must specify the OUTPUTFORMAT in your mapfile, for FlatGeobuf such as:

OUTPUTFORMAT
  NAME "OGRFLATGEOBUF"
  DRIVER "OGR/FlatGeoBuf"
  FORMATOPTION "STORAGE=filesystem"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.fgb"
  FORMATOPTION "LCO:VERIFY_BUFFERS=NO"
END

Next, you must specify that OUTPUTFORMAT using the wfs_getfeature_formatlist metadata parameter, at the LAYER level, such as:

METADATA
  "ows_title"         "World Countries"
  "ows_abstract"      "World Countries, served by MS4W"
  "ows_include_items" "all"
  "gml_include_items" "all"
  "gml_featureid"     "ogc_fid"
  "wfs_getfeature_formatlist" "OGRGML,OGRFLATGEOBUF"
  "ows_geomtype"      "MultiPolygon"
  "wfs_use_default_extent_for_getfeature" "false"
END

Verify that OUTPUTFORMAT through a GetCapilities request, and look for the GetFeature/ResultFormat section, such as:

http://127.0.0.1/cgi-bin/mapserv.exe?
   map=/ms4w/apps/local-demo/flatgeobuf.map
   &SERVICE=wfs
   &VERSION=1.0.0
   &REQUEST=GetCapabilities

Which gives a response containing:

../../_images/wfs-outputformat-flatgeobuf.png

Once we see the OGRFLATGEOBUF in the GetCapabilities response, then we can add &OUTPUTFORMAT=OGRFLATGEOBUF to the end of any WFS GetFeature request, such as:

http://127.0.0.1/cgi-bin/mapserv.exe?
   map=/ms4w/apps/local-demo/flatgeobuf.map
   &SERVICE=WFS
   &REQUEST=GetFeature
   &VERSION=2.0.0
   &TYPENAMES=ms:countries
   &STARTINDEX=0
   &COUNT=1000
   &SRSNAME=urn:ogc:def:crs:EPSG::4326
   &BBOX=-333.02127061643835759,-189,326.66640061643835224,189,urn:ogc:def:crs:EPSG::4326
   &OUTPUTFORMAT=OGRFLATGEOBUF

and a file named result.fgb will be returned.

FlatGeobuf ¶

More Information ¶

File listing ¶

Data Access / Connection Method ¶

Method 1: Direct connection (native access)¶

Method 2: Connect through OGR ¶

Step 1: Use ogrinfo to examine ¶

Step 2: Add the layer in your mapfile ¶

Method 1: Direct connection (native access)¶

Method 2: Connect through OGR ¶

Step 3: Test your Mapfile with map2img ¶

Optional: Configure your WFS service to Output FlatGeoBuf ¶

Navigation

Current Table Of Contents

	Home \| Products \| Issue Tracker \| Service Providers \| FAQ \| Twitter \| Download \|