geotransform

I move a lot of small amount of data regularly, and I don’t like to use big machinery (looking at you FME) for a rather trivial task.

Codeberg repo

Objectives

GDAL/OGR is nice but managing their options and config can be tedious, especially if you have to run many, for example, ogr2ogr commands.

Here we just aim at storing transfer informations into json files and let the tool execute them.

Usage

Usage: gt [OPTIONS] COMMAND [ARGS]...

  Transform data into different formats using OGR and configurations files.

Options:
  --version      Show the version and exit.
  -v, --verbose  Increase output verbosity
  -h, --help     Show this message and exit.

Commands:
  graph     Graph the configuration file(s).
  run       Run the configuration file(s).
  validate  Validate the configuration file(s).

Transforms are configured as follow:

{
  "src": "my_source.connexion",
  "dst": "my_destination.connexion",
  "config": ["config1 VALUE1", "config2 VALUE2", "config3 VALUE3"],
  "options": {"preserveFID": true, "limit": 5, "SQLStatement": "select id, geom from my_table"},
}

A single file can contain multiple transforms stored as a list.

A Transform is defined by at most 4 properties:

  • src and dst are the GDAL source and destination dataset paths, as described by the drivers documentation. They are mandatory.

  • config contains a list of configuration options. Each element contains the key and its value, separated by a space (e.g. ["PG_USE_COPY YES", "OGR_TRUNCATE YES"]).

  • options contains a list of VectorTranslateOptions as described in the gdal python API documentation.

Options are partially validated at the moment, so read carefully what they are.

Subcommands

There are 3 subcommands for now: graph, validate and run.

graph

The graph command creates a visual representation of transforms defined in a configuration file. You’ll need Graphviz installed on your system. Running gt graph -f graph.png config.json on the example json above gives the following figure:

graph

Mostly a visual help for complex configurations.

validate

As its name says, it validates a configuration file and gives hints on badly formed options. Using a “preserveFID” value of 8 instead of a boolean would give:

Invalid option in configuration file: ['preserveFID'] = 8

run

Runs the configuration file. The file is validated before.