Usage with Python projects

After installing Raster Loader, you can use it in your Python project.

First, import the corresponding connection class from the raster_loader package.

For BigQuery, use BigQueryConnection:

from raster_loader import BigQueryConnection

For Snowflake, use SnowflakeConnection:

from raster_loader import SnowflakeConnection

For Databricks, use DatabricksConnection:

from raster_loader import DatabricksConnection

Then, create a connection object with the appropriate parameters.

For BigQuery:

connection = BigQueryConnection('my-project')

Note

Accessing BigQuery with Raster Loader requires the GOOGLE_APPLICATION_CREDENTIALS environment variable to be set to the path of a JSON file containing your BigQuery credentials. See the GCP documentation for more information.

For Snowflake:

connection = SnowflakeConnection('my-user', 'my-password', 'my-account', 'my-database', 'my-schema')

For Databricks:

connection = DatabricksConnection('my-server-hostname', 'my-token', 'my-cluster-id')

Uploading a raster file

To upload a raster file, use the upload_raster function

For example:

connection.upload_raster(
    file_path = 'path/to/raster.tif',
    fqn = 'database.schema.tablename',
)

This function returns True if the upload was successful.

The input raster must be a GoogleMapsCompatible raster. You can make your raster compatible by converting it with the following GDAL command:

gdalwarp -of COG -co TILING_SCHEME=GoogleMapsCompatible -co COMPRESS=DEFLATE -co OVERVIEWS=IGNORE_EXISTINGNONE -co ADD_ALPHA=NO -co RESAMPLING=NEAREST -co BLOCKSIZE=512 <input_raster>.tif <output_raster>.tif

Inspecting a raster file

You can also access and inspect a raster file located in a BigQuery or Snowflake table using the get_records() function.

For example:

records = connection.get_records(
    fqn = 'database.schema.tablename',
)

This function returns a DataFrame with some samples from the raster table (10 rows by default).

See also

See the Module API reference for more details.

To enable compression of the band data, which can significantly reduce storage size, use the compress parameter:

connection.upload_raster(
    file_path = 'path/to/raster.tif',
    fqn = 'database.schema.tablename',
    compress = True,  # Enable gzip compression of band data
    compression_level = 3  # Optional: Set compression level (1-9, default=6)
)

The compression information will be stored in the metadata of the table, and the data will be automatically decompressed when reading it back.