626 lines
26 KiB
Plaintext
626 lines
26 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"# Welcome to EASI <img align=\"right\" src=\"../resources/csiro_easi_logo.png\">\n",
|
|
"\n",
|
|
"This notebook introduces new users to working with EASI notebooks and the Open Data Cube (ODC).\n",
|
|
"\n",
|
|
"It will demonstrate the following basic functionality:\n",
|
|
"- [Notebook setup](#Notebook-setup)\n",
|
|
"- [Select an EASI environment](#Select-an-EASI-environment)\n",
|
|
"- [Connect to the OpenDataCube](#Connect-to-the-OpenDataCube)\n",
|
|
" - [List products](#List-products)\n",
|
|
" - [List measurements and attributes](#List-measurements-and-attributes)\n",
|
|
" - [Choose an area of interest](#Choose-an-area-of-interest)\n",
|
|
" - [Load data](#Load-data)\n",
|
|
" - [Plot the data](#Plot-the-data)\n",
|
|
" - [Masking and scaling](#Masking-and-scaling)\n",
|
|
" - [Perform a calculation on the data](#Perform-a-calculation-on-the-data)\n",
|
|
" - [Save the results to file](#Save-the-results-to-file)\n",
|
|
"- [Summary](#Summary)\n",
|
|
"- [Be a good cloud citizen](#Be-a-good-cloud-citizen)\n",
|
|
"- [Further reading](#Further-reading)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Notebook setup\n",
|
|
"\n",
|
|
"A notebook consists of cells that contain either text descriptions or python code for performing operations on data.\n",
|
|
"\n",
|
|
"1. Start by clicking on the cell below to select it.\n",
|
|
"1. Execute a selected cell, or each cell in sequence, by clicking the ▶ button (in the notebook toolbar above) or pressing `Shift`+`Enter`.\n",
|
|
"1. Each cell will show an asterisk icon <font color='#999'>[*]:</font> when it is running. Once this changes to a number, the cell has finished.\n",
|
|
"1. The cell below imports packages to use and sets some formatting options."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Common imports and settings\n",
|
|
"import os, sys, re\n",
|
|
"from pathlib import Path\n",
|
|
"from IPython.display import Markdown\n",
|
|
"import pandas as pd\n",
|
|
"pd.set_option(\"display.max_rows\", None)\n",
|
|
"import xarray as xr\n",
|
|
"\n",
|
|
"# Datacube\n",
|
|
"import datacube\n",
|
|
"from datacube.utils.aws import configure_s3_access\n",
|
|
"import odc.geo.xr\n",
|
|
"from datacube.utils import masking\n",
|
|
"# https://github.com/GeoscienceAustralia/dea-notebooks/tree/develop/Tools\n",
|
|
"from dea_tools.plotting import display_map, rgb\n",
|
|
"\n",
|
|
"# Basic plots\n",
|
|
"%matplotlib inline\n",
|
|
"# import matplotlib.pyplot as plt\n",
|
|
"# plt.rcParams['figure.figsize'] = [12, 8]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### EASI defaults\n",
|
|
"\n",
|
|
"Each EASI deployment has a different set of products in its opendatacube database. We introduce a set of defaults to allow these training notebooks to be used between EASI deployments.\n",
|
|
"\n",
|
|
"There are also some convenience or helper functions that we use in most notebooks. The cell below can be copied and adapted into your own noteboks."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# EASI defaults\n",
|
|
"# These are convenience functions so that the notebooks in this repository work in all EASI deployments\n",
|
|
"\n",
|
|
"# The `git.Repo()` part returns the local directory that easi-notebooks has been cloned into\n",
|
|
"# If using the `easi-tools` functions from another path, replace `repo` with your local path to `easi-notebooks` directory\n",
|
|
"try:\n",
|
|
" import git\n",
|
|
" repo = git.Repo('.', search_parent_directories=True).working_tree_dir # Path to this cloned local directory\n",
|
|
"except (ImportError, git.InvalidGitRepositoryError):\n",
|
|
" repo = Path.home() / 'easi-notebooks' # Reasonable default\n",
|
|
" if not repo.is_dir():\n",
|
|
" raise RuntimeError('To use `easi-tools` please provide the local path to `https://github.com/csiro-easi/easi-notebooks`')\n",
|
|
"if repo not in sys.path:\n",
|
|
" sys.path.append(str(repo)) # Add the local path to `easi-notebooks` to python\n",
|
|
"\n",
|
|
"from easi_tools import EasiDefaults\n",
|
|
"from easi_tools import initialize_dask, xarray_object_size"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"For this notebook we select the default **Landsat** product."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"easi = EasiDefaults()\n",
|
|
"\n",
|
|
"family = 'landsat'\n",
|
|
"product = easi.product(family)\n",
|
|
"display(Markdown(f'Default {family} product for \"{easi.name}\": [{product}]({easi.explorer}/products/{product})'))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Connect to the OpenDataCube\n",
|
|
"\n",
|
|
"The `Datacube()` API provides search, load and information functions for data products *indexed* in an ODC database. More information on the Open Data Cube software:\n",
|
|
"\n",
|
|
"- https://datacube-core.readthedocs.io/en/latest/\n",
|
|
"- https://github.com/opendatacube/datacube-core"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"dc = datacube.Datacube()\n",
|
|
"\n",
|
|
"# Access AWS \"requester-pays\" buckets\n",
|
|
"# This is necessary for reading data from most third-party AWS S3 buckets such as for Landsat and Sentinel-2\n",
|
|
"configure_s3_access(aws_unsigned=False, requester_pays=True);"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### List products\n",
|
|
"Show all available products in the ODC database and list them along with selected properties.\n",
|
|
"\n",
|
|
"The **ODC Explorer** also has this information and more: view available products, data coverage, product definitions, dimensions, metadata and paths to the files.\n",
|
|
"\n",
|
|
"The product definitions include details about the *measurements* (or bands) in each product and, usually, the spatial resolution and CRS (if common to all member *datasets*).\n",
|
|
"\n",
|
|
"> **Exercise**: Browse the ODC Explorer link and find the information described above."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"display(Markdown(f'#### ODC Explorer: {easi.explorer}'))\n",
|
|
"\n",
|
|
"products = dc.list_products() # Pandas DataFrame\n",
|
|
"products"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### List measurements and attributes\n",
|
|
"\n",
|
|
"The data arrays for each product are called **measurements**. In different data science domains these might also be called the \"bands\", \"variables\" or \"parameters\" of a product.\n",
|
|
"\n",
|
|
"List the measurements of a product. The columns are selected attributes or metadata for each measurement.\n",
|
|
"\n",
|
|
"> **Hint**: Measurements often have **aliases** defined. Any of the available alias names can be used in place of the measurement name when loading (reading) data. The *xarray* variable name will then be the alias name. Use this feature to help make your loaded data more consistent between products. We do this below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"measurements = dc.list_measurements() # Pandas DataFrame, all products\n",
|
|
"measurements.loc[[product]]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Choose an area of interest\n",
|
|
"\n",
|
|
"Choose an area of interest with `latitude`/`longitude` bounds. The `display_map` function will draw a map with the bounding box highlighted. See also the ODC Explorer website for the available *latitude*, *longitude* and *time* ranges for each product.\n",
|
|
"\n",
|
|
"> **Exercise**: Feel free to change the `latitude`/`longitude` or `time` ranges of the query below. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Default area of interest\n",
|
|
"\n",
|
|
"display(Markdown(f\"#### Location: {easi.location}\"))\n",
|
|
"display(Markdown(f\"See: {easi.explorer}/products/{product}\"))\n",
|
|
"\n",
|
|
"latitude = easi.latitude\n",
|
|
"longitude = easi.longitude\n",
|
|
"\n",
|
|
"# Or set your own latitude / longitude\n",
|
|
"# latitude = (-36.3, -35.8)\n",
|
|
"# longitude = (146.8, 147.3)\n",
|
|
"\n",
|
|
"display_map(longitude, latitude)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Load data \n",
|
|
"Here we load product data for a given latitude, longitude and time range. The `datacube.load()` function returns an **xarray.Dataset** object.\n",
|
|
"\n",
|
|
"Once you have an xarray object this can be used with many Python packages. Further information on **xarray**:\n",
|
|
"\n",
|
|
"- https://tutorial.xarray.dev/overview/xarray-in-45-min.html\n",
|
|
"- https://xarray.pydata.org/en/stable/user-guide/data-structures.html\n",
|
|
"\n",
|
|
"**What is the size of my dataset?**\n",
|
|
"\n",
|
|
"The `display(data)` view is a convenient way to check the data request size, shape and attributes.\n",
|
|
"\n",
|
|
"> **Exercise**: Click the various arrows and icons in the *xarray.Dataset* output from the previous cell to reveal information about your data.\n",
|
|
"\n",
|
|
"The `data.nbytes` property returns the number of bytes in the xarray Dataset of DataArray. We have a function that formats this value for convenience.\n",
|
|
"\n",
|
|
"**Datacube.load() notes**\n",
|
|
"\n",
|
|
"- Use `measurements=[measurement or alias names]` to only load the measurements you will use, and label them accordingly.\n",
|
|
"- The `output_crs` and `resolution` parameters allow for remapping to a new grid. These will be required if default values are not defined for the product (see *measurement* attributes).\n",
|
|
"- The `datacube.load()` function does not apply missing values or scaling attributes. These are left to the user's discretion and requirements.\n",
|
|
"- `dask_chunks` will return a **dask** array. See the [EASI tutorial dask notebooks](dask/01_-_Introduction_to_Dask.ipynb) for information and examples.\n",
|
|
"\n",
|
|
"> **Exercise**: The default is to load all available measurements. Load a selected set of measurements or alias names and consider the result.<br>\n",
|
|
"> **Exercise**: The default is to load the data arrays onto a default grid that closely matches the source data. Change the target resolution or CRS and consider the result."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# A standard datacube.load() call.\n",
|
|
"# This may take a few minutes while the data are loaded into JupyterLab (so choose a small area and time range).\n",
|
|
"# Using a dask cluster will make this step seem quicker (see other notebooks for examples).\n",
|
|
"\n",
|
|
"target_crs = easi.crs(family) # If defined, else None\n",
|
|
"target_res = easi.resolution(family) # If defined, else None\n",
|
|
"\n",
|
|
"data = dc.load(\n",
|
|
" product = product,\n",
|
|
" latitude = latitude,\n",
|
|
" longitude = longitude,\n",
|
|
" time = easi.time,\n",
|
|
" measurements = ['red', 'green', 'blue', 'nir'], # List of selected measurement names or aliases\n",
|
|
"\n",
|
|
" output_crs = target_crs, # Target CRS\n",
|
|
" resolution = target_res, # Target resolution\n",
|
|
" # dask_chunks = {'x':2048, 'y':2048), # Dask chunk size. Requires a dask cluster (see the \"dask\" notebooks)\n",
|
|
" group_by = 'solar_day', # Group by day method\n",
|
|
")\n",
|
|
"\n",
|
|
"display(data)\n",
|
|
"\n",
|
|
"display(f'Number of bytes: {data.nbytes}')\n",
|
|
"display(xarray_object_size(data))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Plot the data\n",
|
|
"Plot the measurement data for a set of timesteps. The `xarray.plot()` function can simplify the rendering of plots by using the labelled dimensions and data ranges automatically. \n",
|
|
"\n",
|
|
"See the [EASI tutorial visualisation notebook](03-visualisation.ipynb) for information and examples.\n",
|
|
"\n",
|
|
"- The [robust](https://docs.xarray.dev/en/stable/user-guide/plotting.html#robust) option excludes outliers when calculating the colour limts for a more consistent result across subplots (time layers in this case).\n",
|
|
"\n",
|
|
"> **Exercise**: Change the data variable and perhaps the selection of time layers."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Select a data variable (measurement) name\n",
|
|
"band = 'nir'\n",
|
|
"\n",
|
|
"# Xarray simple array plotting\n",
|
|
"display(Markdown(f'#### Measurement: {band}'))\n",
|
|
"data[band].plot(col=\"time\", robust=True, col_wrap=4);"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Masking and scaling\n",
|
|
"\n",
|
|
"Most data products include a no-data value and/or a *data quality* array that can be used to mask (filter) the measurement arrays. For example, remote sensing quality arrays often include a \"cloud\" confidence flag that can be used to remove pixels affected by clouds from further analysis. Measurement arrays can also include *scale and offset factors* to transform the array values to scientific values.\n",
|
|
"\n",
|
|
"This step is common to most data analysis problems so we encourage users to find and understand the relevant quality, scale and offset metadata for each product used and apply these in their applications. For example, here are the relevant product metadata pages for Landsat and Sentinel-2:\n",
|
|
"- https://www.usgs.gov/landsat-missions/landsat-science-products\n",
|
|
"- https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/level-2a/algorithm\n",
|
|
"\n",
|
|
"The opendatacube provides functions for creating mask arrays from quality measurements defined in the *product definition*. These are covered in various product-specific and example notebooks.\n",
|
|
"\n",
|
|
"Here we use a simple function to create a mask using only the *no data* value of each measurement array.\n",
|
|
"\n",
|
|
"> **Exercise**: What is the effect of applying, or not applying, the `valid_mask` to these data (hint: see the NDVI plot below)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Mask by nodata\n",
|
|
"\n",
|
|
"# Under the hood: data != data.nodata -> bool\n",
|
|
"# Applies to each variable in an xarray.Dataset (including any bit-masks)\n",
|
|
"valid_mask = masking.valid_data_mask(data)\n",
|
|
"\n",
|
|
"# Use numpy.where() to apply a mask array to measurement arrays\n",
|
|
"valid_data = data.where(valid_mask) # Default: Where False replace with NaN -> convert dtype to float64\n",
|
|
"\n",
|
|
"# Or provide a no-data value and retain the dtype\n",
|
|
"# nodata = -9999 # A new nodata value\n",
|
|
"# valid_data = data.where(valid_mask, nodata) # Where False replace with nodata -> retain dtype if compatible"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Perform a calculation on the data\n",
|
|
"As a simple example, we calculate the [Normalized Difference Vegetation Index (NDVI)](https://en.wikipedia.org/wiki/Normalized_difference_vegetation_index) using the *near infra-red (NIR)* and *red* measurements of the product.\n",
|
|
"\n",
|
|
"- **Note**: This may not be a realistic *NDVI* example if the measurements have not been scaled to science values.\n",
|
|
"\n",
|
|
"See this DEA notebook for a set of other remote sensing band indices: https://github.com/GeoscienceAustralia/dea-notebooks/blob/develop/Tools/dea_tools/bandindices.py\n",
|
|
"\n",
|
|
"> **Exercise**: Calculate a different remote sensing band index, possibly with different measurements loaded into `data`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Get measurement or alias names corresponding to near-infra read (NIR) and Red bands.\n",
|
|
"\n",
|
|
"# Calculate the NDVI\n",
|
|
"varname = 'ndvi'\n",
|
|
"band_diff = valid_data.nir - valid_data.red\n",
|
|
"band_sum = valid_data.nir + valid_data.red\n",
|
|
"calculation = band_diff / band_sum # xarray.DataArray\n",
|
|
"\n",
|
|
"# Convert to an xarray.Dataset\n",
|
|
"calculation = calculation.to_dataset(name=varname, promote_attrs=True)\n",
|
|
"\n",
|
|
"# Plot the NDVI\n",
|
|
"display(Markdown(f'#### Calculation: {varname.upper()}'))\n",
|
|
"calculation[varname].plot(col=\"time\", robust=True, col_wrap=4, vmin=0, vmax=0.7, cmap='RdYlGn');"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Save the results to file\n",
|
|
"We can save an `xarray.Dataset` to a file(s) that can then be imported into other applications for further analysis or publication if required.\n",
|
|
"\n",
|
|
"In the code below, the file(s) will be saved to your home directory and appear in the File Browser panel to the left. You may need to select the `folder` icon to go to the top level (`$HOME`) and then `output/`.\n",
|
|
"\n",
|
|
"Download a file by `'right-click' Download`.\n",
|
|
"\n",
|
|
"> **Exercise**: Use the Terminal to also list the files in the output directory."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### To netCDF"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Xarray can save the data to a netCDF file\n",
|
|
"# See also: https://github.com/GeoscienceAustralia/dea-notebooks/blob/develop/Frequently_used_code/Exporting_NetCDFs.ipynb\n",
|
|
"\n",
|
|
"target = f'{os.environ[\"HOME\"]}/output'\n",
|
|
"if not os.path.isdir(target):\n",
|
|
" os.mkdir(target)\n",
|
|
"\n",
|
|
"calculation.time.attrs.pop('units', None) # Xarray re-applies this\n",
|
|
"calculation.to_netcdf(f'{target}/example_landsat_{varname}.nc')\n",
|
|
"calculation.close()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### To Cloud-optimised Geotiff files"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Single-layer time slices can be written to Geotiff files\n",
|
|
"# See also: https://github.com/GeoscienceAustralia/dea-notebooks/blob/develop/Frequently_used_code/Exporting_GeoTIFFs.ipynb\n",
|
|
"\n",
|
|
"target = f'{os.environ[\"HOME\"]}/output'\n",
|
|
"if not os.path.isdir(target):\n",
|
|
" os.mkdir(target)\n",
|
|
"\n",
|
|
"for i in range(len(calculation.time)):\n",
|
|
" date = calculation[varname].isel(time=i).time.dt.strftime('%Y%m%d').data\n",
|
|
" single = calculation[varname].isel(time=i)\n",
|
|
" single.odc.write_cog(\n",
|
|
" fname=f'{target}/example_landsat_{varname}_{date}.tif',\n",
|
|
" overwrite=True,\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Summary\n",
|
|
"\n",
|
|
"This notebook introduced the main steps for querying data (with OpenDataCube), and filtering, plotting, calculating and saving a \"cube\" of data (with **Xarray**).\n",
|
|
"\n",
|
|
"There is plenty of detail and options to explore so please work through the other notebooks to learn more and refer back to these notebooks when required. We encourage you to create or bring your own notebooks, and adapt notebooks from other [open-license repositories](https://docs.asia.easi-eo.solutions/user-guide/users-guide/03-using-notebooks/#other-available-odc-notebooks)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"rgb(valid_data.isel(time=2), ['red', 'green', 'blue'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Be a good cloud citizen\n",
|
|
"\n",
|
|
"It is good practice to close your JupyterLab session when you have finished with it. Your home directory will be retained and in most cases your workspace of open notebooks will also be retained. These will be available when you return to EASI JupyterLab.\n",
|
|
"\n",
|
|
"Select `File` menu and `Hub Control Panel` from the JupyterLab menu. Then `Stop My Server`.\n",
|
|
"- Stop My Server: Your JupyterLab resources will be safely shutdown.\n",
|
|
"- Log Out: Log out of your JupyterLab *browser* session. Your JupyterLab resources will remain active until the system cleans up.\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Further reading \n",
|
|
"\n",
|
|
"#### JupyterLab\n",
|
|
"The JupyterLab website has excellent documentation and video instructions. We recommend users take a few minutes to orientate themselves with the use and features of JupyterLab.\n",
|
|
"\n",
|
|
"> *Recommended level: Familiarity with notebooks.*\n",
|
|
"\n",
|
|
"- Getting started: [https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html](https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html)\n",
|
|
"- Drag and drop upload of files: [https://jupyterlab.readthedocs.io/en/stable/user/files.html](https://jupyterlab.readthedocs.io/en/stable/user/files.html)\n",
|
|
"\n",
|
|
"#### Python3\n",
|
|
"There are many options for learning Python from online resources or facilitated training. Some examples are offered here with no suggestion that EASI endorses any of them.\n",
|
|
"\n",
|
|
"> *Recommended level: Basic Python knowledge and familiarity with array manipulations, __numpy__ and __xarray__. Familiarity with some plotting libraries (e.g., __matplotlib__) would also help.*\n",
|
|
"\n",
|
|
"- Get started: [https://www.python.org/about/gettingstarted](https://www.python.org/about/gettingstarted/)\n",
|
|
"- Learn Python tutorials: [https://www.learnpython.org](https://www.learnpython.org/)\n",
|
|
"- Data Camp: [https://www.datacamp.com](https://www.datacamp.com/)\n",
|
|
"- David Beazley courses: [https://dabeaz-course.github.io/practical-python](https://dabeaz-course.github.io/practical-python/)\n",
|
|
"- Numpy: [https://numpy.org/doc/stable/user/quickstart.html](https://numpy.org/doc/stable/user/quickstart.html)\n",
|
|
"- Xarray: [http://xarray.pydata.org/en/stable/user-guide/data-structures.html](http://xarray.pydata.org/en/stable/user-guide/data-structures.html)\n",
|
|
"- Pandas: [https://pandas.pydata.org/docs/getting_started/index.html](https://pandas.pydata.org/docs/getting_started/index.html)\n",
|
|
"\n",
|
|
"#### Git\n",
|
|
"Git is a document version control system. It retains a full history of changes to all files (including deleted ones) by tracking incremental changes and recording a history timeline of changes. The best way to learn Git is by practice and incrementally: start with simple, common actions and gain more knowledge as required. \n",
|
|
"\n",
|
|
"> *Recommended level: Basic understanding of Git repositories (e.g., github.com) and practices such as __clone__, __pull__/__push__ and __merging__ changes.*\n",
|
|
"\n",
|
|
"- Getting started: [https://git-scm.com/doc](https://git-scm.com/doc)\n",
|
|
"- JupyterLab Git extension: [https://github.com/jupyterlab/jupyterlab-git#readme](https://github.com/jupyterlab/jupyterlab-git#readme)\n",
|
|
"- DEA Git guide: [https://github.com/GeoscienceAustralia/dea-notebooks/wiki/Guide-to-using-DEA-Notebooks-with-git](https://github.com/GeoscienceAustralia/dea-notebooks/wiki/Guide-to-using-DEA-Notebooks-with-git)\n",
|
|
"- Undoing things guide: [https://git-scm.com/book/en/v2/Git-Basics-Undoing-Things](https://git-scm.com/book/en/v2/Git-Basics-Undoing-Things)\n",
|
|
"- Understanding branches: [https://nvie.com/posts/a-successful-git-branching-model](https://nvie.com/posts/a-successful-git-branching-model)\n",
|
|
"\n",
|
|
"#### Open Data Cube\n",
|
|
"The ODC is a Python library that allows the user to search for datasets in its database and return an **xarray** data array. There are convenience functions and methods for resampling, reprojecting and masking the data.\n",
|
|
"\n",
|
|
"> *Recommended level: Overview of the design and intent of ODC or other datacubes*\n",
|
|
"\n",
|
|
"- ODC website: [https://www.opendatacube.org](https://www.opendatacube.org)\n",
|
|
"- ODC API reference: [https://datacube-core.readthedocs.io](https://datacube-core.readthedocs.io)\n",
|
|
"- ODC Github code: [https://github.com/opendatacube](https://github.com/opendatacube)\n",
|
|
"\n",
|
|
"#### Notebooks for EO data analysis \n",
|
|
"There are growing collections of notebooks available from many organizations, most of which can be adapted to use with ODC and EASI.\n",
|
|
"\n",
|
|
"> *Recommended level: Overview of available notebooks and selected EO applications*\n",
|
|
"\n",
|
|
"- CSIRO EASI notebooks: [https://github.com/csiro-easi/easi-notebooks](https://github.com/csiro-easi/easi-notebooks)\n",
|
|
"- Digital Earth Australia: [https://github.com/GeoscienceAustralia/dea-notebooks](https://github.com/GeoscienceAustralia/dea-notebooks)\n",
|
|
"- Digital Earth Africa: [https://github.com/digitalearthafrica/deafrica-sandbox-notebooks](https://github.com/digitalearthafrica/deafrica-sandbox-notebooks)\n",
|
|
"- CEOS SEO (NASA): [https://github.com/ceos-seo/data_cube_notebooks](https://github.com/ceos-seo/data_cube_notebooks)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|