# Vector Data

RasterFrames provides a variety of ways to work with spatial vector data (points, lines, and polygons) alongside raster data.

 * DataSource for GeoJSON format
 * Ability to convert between from [GeoPandas][GeoPandas] and Spark DataFrames
 * In PySpark, geometries are [Shapely][Shapely] objects, providing a great deal of interoperability
 * Many Spark functions for working with columns of geometries
 * Vector data is also the basis for @ref:[zonal map algebra](zonal-algebra.md) operations.

```python, setup, echo=False
import pyrasterframes
from pyrasterframes.utils import create_rf_spark_session
import pyrasterframes.rf_ipython
import geopandas
import folium
spark = create_rf_spark_session('local[2]')
```

## GeoJSON DataSource

```python, read_geojson
from pyspark import SparkFiles
admin1_us_url = 'https://raw.githubusercontent.com/datasets/geo-admin1-us/master/data/admin1-us.geojson'
spark.sparkContext.addFile(admin1_us_url)  # this lets us read http scheme uri's in spark

df = spark.read.geojson(SparkFiles.get('admin1-us.geojson'))
df.printSchema()
```

The properties of each discrete geometry are available as columns of the DataFrame, along with the geometry itself.

## GeoPandas and RasterFrames

You can also convert a [GeoPandas][GeoPandas] GeoDataFrame to a Spark DataFrame, preserving the geometry column. This means that any vector format that can be read with [OGR][OGR] can be converted to a Spark DataFrame. In the example below, we expect the same schema as the DataFrame defined above by the GeoJSON reader. Note that in a GeoPandas DataFrame there can be heterogeneous geometry types in the column, which may fail Spark's schema inference.

```python, read_and_normalize
import geopandas
from shapely.geometry import MultiPolygon

def poly_or_mp_to_mp(g):
    """ Normalize polygons or multipolygons to all be multipolygons. """
    if isinstance(g, MultiPolygon):
        return g
    else:
        return MultiPolygon([g])

gdf = geopandas.read_file(admin1_us_url)
gdf.geometry = gdf.geometry.apply(poly_or_mp_to_mp)
df2 = spark.createDataFrame(gdf)
df2.printSchema()
```

## Shapely Geometry Support

The `geometry` column will have a Spark user-defined type that is compatible with [Shapely][Shapely] when working with Python via PySpark. This means that when the data is collected to the driver, it will be a Shapely geometry object.

```python, show_geom
the_first = df.first()
print(type(the_first['geometry']))
```

Since it is a geometry we can do things like this:

```python, show_wkt
the_first['geometry'].wkt
```

You can also write user-defined functions that take geometries as input, output, or both, via user defined types in the [geomesa_pyspark.types](https://github.com/locationtech/rasterframes/blob/develop/pyrasterframes/src/main/python/geomesa_pyspark/types.py) module. Here is a simple **but inefficient** example of a user-defined function that uses both a geometry input and output to compute the centroid of a geometry. Observe in a sample of the data the geometry columns print as well known text (wkt).

```python, add_centroid
from pyspark.sql.functions import udf
from geomesa_pyspark.types import PointUDT

@udf(PointUDT())
def inefficient_centroid(g):
    return g.centroid

df.select(df.state_code, inefficient_centroid(df.geometry))
```

## GeoMesa Functions and Spatial Relations

As documented in the @ref:[function reference](reference.md), various user-defined functions implemented by GeoMesa are also available for use. The example below uses a GeoMesa user-defined function to compute the centroid of a geometry. It is logically equivalent to the example above, but more efficient.

```python, native_centroid
from pyrasterframes.rasterfunctions import st_centroid
df.select(df.state_code, inefficient_centroid(df.geometry), st_centroid(df.geometry))
```

The RasterFrames vector functions and GeoMesa functions also provide a variety of spatial relations that are useful in combination with the geometric properties of projected rasters. In this example, we use the @ref:[built-in Landsat catalog](raster-catalogs.md#using-built-in-experimental-catalogs) which provides an extent. We will convert the extent to a polygon and filter to those within approximately 50 km of a selected point.

```python, spatial_relation, evaluate=True
from pyrasterframes.rasterfunctions import st_geometry, st_bufferPoint, st_intersects, st_point
from pyspark.sql.functions import lit
l8 = spark.read.format('aws-pds-l8-catalog').load()

l8 = l8.withColumn('geom', st_geometry(l8.bounds_wgs84))  # extent to polygon
l8 = l8.withColumn('paducah', st_point(lit(-88.628), lit(37.072)))  # col of points

l8_filtered = l8 \
                .filter(st_intersects(l8.geom, st_bufferPoint(l8.paducah, lit(50000.0)))) \
                .filter(l8.acquisition_date > '2018-02-01') \
                .filter(l8.acquisition_date < '2018-03-11')
```

```python, folium, echo=False
geo_df = geopandas.GeoDataFrame(
    l8_filtered.select('geom', 'bounds_wgs84').toPandas(),
    crs='EPSG:4326',
    geometry='geom')

# display as folium / leaflet map
m = folium.Map()
layer = folium.GeoJson(geo_df.to_json())
m.fit_bounds(layer.get_bounds())
m.add_child(layer)
m
```

[GeoPandas]: http://geopandas.org
[OGR]: https://gdal.org/drivers/vector/index.html
[Shapely]: https://shapely.readthedocs.io/en/latest/manual.html