17  Install DuckDB Extensions via HTTPS

If your network blocks HTTP requests, you cannot simply install DuckDB extensions by e.g. INSTALL spatial; (it will fail after ~10 minutes), because of the dependency on httpfs, an extension itself – see details here.

The workaround is to separately download the httpfs extension (see here for the list of architectures, even though within Databricks, the below preset is correct):

%pip install duckdb --quiet

import duckdb
import os
from urllib.parse import urlparse
import requests

ARCHITECTURE = "linux_amd64"
duckdb_version = duckdb.__version__
url = f"https://extensions.duckdb.org/v{duckdb_version}/{ARCHITECTURE}/httpfs.duckdb_extension.gz"

output_file = os.path.basename(urlparse(url).path)
response = requests.get(url, timeout=30)
response.raise_for_status()
with open(output_file, "wb") as f:
    f.write(response.content)

duckdb.install_extension(output_file)

os.remove(output_file)

duckdb.sql("SET custom_extension_repository='https://extensions.duckdb.org'")

And now you can install other extensions, such as:

duckdb.sql("install spatial; load spatial")