Maron, M. - Visualisation and mapping of building open data - Mikel Maron, Mapbox.

Visualisation and mapping of building open data - Mikel Maron, Mapbox.
Workshop on Modernising Statistical Systems, OECD. 2018

1. OECD Working Party on Territorial Indicators Workshop
Modernising statistical systems for better data on regions and cities

Open Buildings: Visualization and
Conflation at Scale

Mikel Maron
Mapbox | Community Team
[email protected]

2. Introduce Mapbox
Building Conflation
Building Visualization

6. https://mapbox.com/community/

8. What do we do?
Connect
Respond to incoming inquiries,
attend events, and reach out to
target partners.
Assess
Identify good-fit projects
and scope their needs.
Match
Connect projects to the
support they need and
facilitate the relationship.
Our Tools & Data
Our Operations
Our Community
Partnerships
Share
Evaluate, assess impact,
promote, celebrate

9. Build with empathy

10. Visualization

11. visualization

16. Telemetry

17. Passive telemetry

18. Telemetry Deep Dive

20. telemetry: 250 million miles every day

21. Analysis

24. Imagery

29. VisionSDK

30. VisionSDK

31. VisionSDK

32. Logistics

33. Logistics

34. Logistics

35. Community Generated Data

40. Building Conflation

41. R&D Goal:
find most efficient way to iterate quickly to
best of both OSM and Microsoft buildings

42. In the US
Microsoft: ~125 million
OSM: ~25 million
(~300 million globally)

43. Failed Approaches
●PostGIS: query timed out
●PySpark w/ shapely lib: logic worked, but ~7 days
●GeoSpark: limited functionality, no performance
gain

44. STARK
A framework for Spatio-Temporal Data Analytics on Spark
https://github.com/dbis-ilm/stark

●Binary Space Partitioning
●RTree Indexing
●Spatial Join on Intersects

45. Data Preparation
●Query internal validated OSM data store (in
dynamodb); cost based binary space partitioning
into in 256 partitions
●Pull down Microsoft building data; and partition into
the same
●r-tree index of each partition
●Pair are then distributed through spark cluster.
Uniform partitions result in more efficient operation

46. Operation

●For each Microsoft building polygon
○Drop if bad geometry, or in manual maintained list
of non-buildings
○If Microsoft building intersects OSM, choose OSM
(OSM data assumed better). Otherwise Microsoft
●Apply unique id
○O-{osm_id}-{osm_part_id}
○M-{incrementing_integer_id}

47. Output

●take resulting output, and renders
GeoJSON to s3
●push to tippecanoe, generate mbtiles

48. STARK
USA: 1 Hour 35 minutes -> 125M Microsoft
Buildings + 26M OSM Buildings -> $7 Cost

49. 2 hours

50. Process Evaluation
●Pan and scan samples
●All buildings present from OSM
●Buildings present on tile edges
●Attributes present
●Buildings are extrudable

51. Quality Evaluation
●50 randomly selected Z17 tiles in 65 US metropolitan statistical areas
(MSA)
●Pan and Scan Samples against imagery
○Check for false positives and missing buildings
○Shape approximate shape of representative buildings
○Logical placement of buildings (i.e., buildings not on Roadways)
●Score quality on 1 (High) -> 5 (Low)

Conclusion: MSFT pretty good, some areas better than
others

52. Building Visualization

53. VisionSDK

54. VisionSDK

55. VisionSDK

56. VisionSDK

57. VisionSDK

58. VisionSDK

59. VisionSDK

60. OECD Working Party on Territorial Indicators Workshop
Modernising statistical systems for better data on regions and cities

Open Buildings: Visualization and
Conflation at Scale

Mikel Maron
Mapbox | Community Team
[email protected]