Help ?¶
For each and every command, there is a --help option. Please consult it to
grasp the details for a command, its arguments and optional parameters, default
values and settings that can further shape the output.
For example,
Usage: rekx [OPTIONS] COMMAND [ARGS]...
๐พ ๐ฆ Rekx command line interface prototype
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --version Show the version and exit. โ
โ --log Enable logging. โ
โ --install-completion Install completion for the current shell. โ
โ --show-completion Show completion for the current shell, to copy โ
โ it or customize the installation. โ
โ --help Show this message and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Diagnose chunking layout โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ inspect Inspect Xarray-supported data โ
โ shapes Diagnose chunking shapes in multiple โ
โ Xarray-supported data โ
โ validate-json Validate chunk size consistency along multiple โ
โ Kerchunk reference files How to get available โ
โ variables? โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Suggest chunking layout โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ suggest Suggest a good chunking shape, ex. โ
โ '8784,2600,2600' Needs a review! โ
โ suggest-alternative Suggest a good chunking shape Merge to suggest โ
โ suggest-symmetrical Suggest a good chunking shape Merge to suggest โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Rechunk data โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ modify-chunks Modify in-place the chunk size metadata in NetCDF โ
โ files Yet not implemented! โ
โ rechunk Rechunk NetCDF file โ
โ rechunk-generator Generate variations of rechunking commands for โ
โ multiple files โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Create references โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ reference Create Kerchunk JSON reference files โ
โ reference-parquet Create Parquet references to an HDF5/NetCDF file โ
โ Merge to reference โ
โ reference-multi-parquet Create Parquet references to multiple HDF5/NetCDF โ
โ files Merge to reference-parquet โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Combine references โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ combine Combine Kerchunk reference sets (JSONs to JSON) โ
โ combine-to-parquet Combine Kerchunk reference sets into a single โ
โ Parquet store (JSONs to Parquet) โ
โ combine-parquet-stores Combine multiple Parquet stores (Parquets to โ
โ Parquet) โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Select from time series โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ clip %< Select time series over a location โ
โ select ๎ซฑ Select time series over a location โ
โ select-fast ๎ซฑ Bare read time series from Xarray-supported โ
โ data and optionally write to CSV โฒ Performance โ
โ Test โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Select from Kerchunk references โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ select-json ๎ซฑ Select time series over a location from a JSON โ
โ Kerchunk reference set โ
โ select-parquet ๎ซฑ Select data from a Parquet references store โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ โฒ Read performance โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ read-performance ๎ซฑ Measure read & load in-memory operations of a โ
โ point time series from Xarray-supported data โ
โ read-performance-area ๎ซฑ Measure read & load in-memory operations of an โ
โ area time series from Xarray-supported data โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
The help for the command shapes
Usage: rekx shapes [OPTIONS] SOURCE_DIRECTORY
Diagnose chunking shapes in multiple Xarray-supported data
โญโ Time series โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ * source_directory PATH Source directory path [default: None] โ
โ [required] โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --pattern TEXT Filename pattern โ
โ to match โ
โ [default: *.nc] โ
โ --variable-set [all|coordinates Set of Xarray โ
โ |coordinates-wit variables to โ
โ hout-data|data|m diagnose โ
โ etadata|time] [default: all] โ
โ --validate-consiโฆ --no-validate-cโฆ [default: โ
โ no-validate-consโฆ โ
โ --common-shapes --no-common-shaโฆ Report common โ
โ maximum chunking โ
โ shape โ
โ [default: โ
โ no-common-shapes] โ
โ --help Show this message โ
โ and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Input / Output โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --csv PATH CSV output filename [default: None] โ
โ --verbose -v INTEGER Show details while executing commands โ
โ [default: 0] โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
Verbosity¶
Most of the commands feature an extra --verbose or shortly -v flag. It'll
make rekx to be more communicative about what he did.
For example
check the difference of executing the same command
without -v
and with -v
๐ด Variables are not consistently shaped across all files!
SIS
Variable Shape Files
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
SIS 1 x 1 x 2600 SISin202001010000004231000101MA.nc
SIS 1 x 2600 x 2600 SISin200001010000004231000101MA_1_2600_2600.nc
Logging¶
rekx is growing and learning as we all do, by trial & error :-).
To get some background information on how rekx is crunching data,
we can instruct the --log option right before any subcommand :
SISin202001010000004231000101MA.nc
Vโฆ Shape Chunks Cโฆ Elโฆ Preโฆ โฆ Compโฆ Shโฆ Reaโฆ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Sโฆ 48 x 2600โฆ 1 x 1 โฆ 1โฆ 10โฆ 0.75 - zlib Faโฆ 0.0โฆ
File size: 181550165 bytes, Dimensions: time: 48, lon: 2600, bnds: 2, lat: 2600
* Cache: Size in bytes, Number of elements, Preemption ranging in [0, 1]
A .log is created
containing timestamped details
on the execution of important commands and their output.
Example :