We're updating the issue view to help you get more done. 

Speed up fixing shapefiles

Description

In 3 ways we can fix invalid shapes.
I created unit tests for all 3.
Using the same input file: a 700k polygon shapefile with 28 invalid shapes: 'Polygon must be clockwise'

utils.FixUpShapes
Needed time:

1 00:02:06.6828228

sf.FixUpShapes2
Needed time:

1 00:01:50.2094266

shp.FixUp
Needed time:

1 00:00:38.3133595

In my unit test for shp.FixUp I used Parallel.ForEach:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 var stack = new ConcurrentStack<Shape>(); // Partition the entire source array. var rangePartitioner = Partitioner.Create(0, sf.NumShapes); // Loop over the partitions in parallel. Parallel.ForEach(rangePartitioner, (range, loopState) => { // Loop over each range element without a delegate invocation. for (var i = range.Item1; i < range.Item2; i++) { var shp = sf.Shape[i]; Shape fixedShape; shp.FixUp(out fixedShape); stack.Push(fixedShape); } }); // Create output: var sfOutput = sf.Clone(); foreach (var shape in stack) { sfOutput.EditAddShape(shape); } sfOutput.SaveAs(sfOutputLocation);

In this code I don't copy the fields, that needs to be improved.

Further investigation is needed if we can use Parallel.ForEach with a rangePartitioner in C++ as well.

Status

Assignee

Paul Meems

Reporter

Paul Meems

Labels

None

Epic Link

Fix versions

Affects versions

v5.1.1.0

Priority

Medium