disdrodb.routines package

disdrodb.routines package#

Submodules#

disdrodb.routines.l0 module#

Implement DISDRODB L0 processing.

disdrodb.routines.l0.run_l0a_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0A processing of a specific DISDRODB station when invoked from the terminal.

This function is intended to be called through the disdrodb_run_l0a_station command-line interface.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously with each process using a single thread. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only the first 3 raw data files will be processed. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.l0.run_l0b_station(data_source, campaign_name, station_name, remove_l0a: bool = False, force: bool = False, verbose: bool = True, parallel: bool = True, debugging_mode: bool = False, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0B processing of a specific DISDRODB station when invoked from the terminal.

This function is intended to be called through the disdrodb_run_l0b_station command-line interface.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously, with each process using a single thread to avoid issues with the HDF/netCDF library. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only 100 rows sampled from 3 L0A files will be processed. The default value is False.
remove_l0a (bool, optional) – Whether to remove the processed L0A files. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.l0.run_l0c_station(data_source, campaign_name, station_name, remove_l0b: bool = False, force: bool = False, verbose: bool = True, parallel: bool = True, debugging_mode: bool = False, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0C processing of a specific DISDRODB station when invoked from the terminal.

The DISDRODB L0A and L0B routines just convert source raw data into netCDF format. The DISDRODB L0C routine ingests L0B files and performs data homogenization. The DISDRODB L0C routine takes care of:

removing duplicated timesteps across files,
merging/splitting files into daily files,
regularizing timesteps for potentially trailing seconds,
ensuring L0C files with unique sample intervals.

Duplicated timesteps are automatically dropped if their variable values coincides, otherwise an error is raised.

This function is intended to be called through the disdrodb_run_l0c_station command-line interface.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously, with each process using a single thread to avoid issues with the HDF/netCDF library. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only the first 3 files will be processed. The default value is False.
remove_l0b (bool, optional) – Whether to remove the processed L0B files. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.l1 module#

Implement DISDRODB L1 processing.

disdrodb.routines.l1.define_l1_logs_filename(campaign_name, station_name, start_time, end_time, temporal_resolution)[source][source]#: Define L1 logs filename.

disdrodb.routines.l1.run_l1_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = True, parallel: bool = True, debugging_mode: bool = False, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L1 processing of a specific DISDRODB station when invoked from the terminal.

The L1 routines just filter the raw drop spectrum and compute basic statistics. The L1 routine expects as input L0C files where each file has a unique sample interval.

This function is intended to be called through the disdrodb_run_l1_station command-line interface.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously, with each process using a single thread to avoid issues with the HDF/netCDF library. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only the first 3 files will be processed. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.l2 module#

Implements routines for DISDRODB L2 processing.

disdrodb.routines.l2.define_l2e_logs_filename(campaign_name, station_name, start_time, end_time, temporal_resolution)[source][source]#: Define L2E logs filename.

disdrodb.routines.l2.define_l2m_logs_filename(campaign_name, station_name, start_time, end_time, model_name, temporal_resolution)[source][source]#: Define L2M logs filename.

disdrodb.routines.l2.run_l2e_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = True, parallel: bool = True, debugging_mode: bool = False, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Generate the L2E product of a specific DISDRODB station when invoked from the terminal.

This function is intended to be called through the disdrodb_run_l2e_station command-line interface.

This routine generates L2E files. Files are defined based on the DISDRODB archive settings options. The DISDRODB archive settings allows to produce L2E files either per custom block of time (i.e day/month/year) or per blocks of (rainy) events.

For stations with varying measurement intervals, DISDRODB defines a separate list of partitions for each measurement interval option. In other words, DISDRODB does not mix files with data acquired at different sample intervals when resampling the data.

L0C product generation ensure creation of files with unique sample intervals.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously, with each process using a single thread to avoid issues with the HDF/netCDF library. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only the first 3 files will be processed. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.l2.run_l2m_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = True, parallel: bool = True, debugging_mode: bool = False, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2M processing of a specific DISDRODB station when invoked from the terminal.

This function is intended to be called through the disdrodb_run_l2m_station command-line interface.

Parameters:

data_source (str) – The name of the institution (for campaigns spanning multiple countries) or the name of the country (for campaigns or sensor networks within a single country). Must be provided in UPPER CASE.
campaign_name (str) – The name of the campaign. Must be provided in UPPER CASE.
station_name (str) – The name of the station.
force (bool, optional) – If True, existing data in the destination directories will be overwritten. If False (default), an error will be raised if data already exists in the destination directories.
verbose (bool, optional) – If True (default), detailed processing information will be printed to the terminal. If False, less information will be displayed.
parallel (bool, optional) – If True, files will be processed in multiple processes simultaneously, with each process using a single thread to avoid issues with the HDF/netCDF library. If False (default), files will be processed sequentially in a single process, and multi-threading will be automatically exploited to speed up I/O tasks.
debugging_mode (bool, optional) – If True, the amount of data processed will be reduced. Only the first 3 files will be processed. The default value is False.
data_archive_dir (str, optional) – The base directory of DISDRODB, expected in the format <...>/DISDRODB. If not specified, the path specified in the DISDRODB active configuration will be used.

disdrodb.routines.options module#

Implements ProcessingOption class for DISDRODB routines.

class disdrodb.routines.options.L1ProcessingOptions(filepaths, parallel, sensor_name, temporal_resolutions=None)[source][source]#

Bases: object

Define L1 product processing options.

Define DISDRODB L1 product processing options.

get_folder_partitioning(temporal_resolution)[source][source]#: Return the folder partitioning for a specific L2E product.

get_product_options(temporal_resolution)[source][source]#: Return product options dictionary for a specific L2E product.

group_files_by_temporal_partitions(temporal_resolution)[source][source]#: Return files partitions dictionary for a specific L2E product.

class disdrodb.routines.options.L2ProcessingOptions(product, filepaths, parallel, temporal_resolution)[source][source]#

Bases: object

Define L2 products processing options.

Define DISDRODB L2 products processing options.

disdrodb.routines.options.check_availability_radar_simulations(options)[source][source]#: Check radar simulations are possible for L2E and L2M products.

disdrodb.routines.options.get_model_options(product, model_name)[source][source]#: Return DISDRODB L2M product model options.

disdrodb.routines.options.get_product_options(product, temporal_resolution=None, sensor_name=None)[source][source]#

Return DISDRODB product options.

If temporal resolution is not provided, it returns the global product option. If temporal resolution is provided, it provides the custom product options, if specified. If product=”L1” and sensor_name is specified, it customize product options also by sensor.

disdrodb.routines.options.get_product_temporal_resolutions(product)[source][source]#: Return DISDRODB products temporal resolutions.

disdrodb.routines.options.validate_product_configuration(products_configs_dir)[source][source]#: Validate the DISDRODB products configuration files.

disdrodb.routines.wrappers module#

DISDRODB CLI routine wrappers.

disdrodb.routines.wrappers.create_summary(data_sources=None, campaign_names=None, station_names=None, parallel=False, temporal_resolution='1MIN', data_archive_dir=None, metadata_archive_dir=None)[source][source]#

Create summary figures and tables for a set of DISDRODB station.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.
temporal_resolution (str) – Temporal resolution of the summary. The default value is 1MIN.

disdrodb.routines.wrappers.create_summary_station(data_source, campaign_name, station_name, parallel=False, temporal_resolution='1MIN', data_archive_dir=None)[source][source]#: Create summary figures and tables for a DISDRODB station.

disdrodb.routines.wrappers.run_l0(data_sources=None, campaign_names=None, station_names=None, l0a_processing: bool = True, l0b_processing: bool = True, l0c_processing: bool = True, remove_l0a: bool = False, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0 processing of DISDRODB stations.

This function allows to launch the processing of many DISDRODB stations with a single command. From the list of all available DISDRODB stations, it runs the processing of the stations matching the provided data_sources, campaign_names and station_names.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
l0a_processing (bool) – Whether to launch processing to generate L0A Apache Parquet file(s) from raw data. The default value is True.
l0b_processing (bool) – Whether to launch processing to generate L0B netCDF4 file(s) from L0A data. The default value is True.
l0c_processing (bool) – Whether to launch processing to generate L0C netCDF4 file(s) from L0B data. The default value is True.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
remove_l0b (bool) – Whether to remove the L0B files after having produced all L0C netCDF files. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0A, it processes just the first 3 raw data files. For L0B, it processes 100 rows sampled from 3 L0A files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0_station(data_source, campaign_name, station_name, l0a_processing: bool = True, l0b_processing: bool = True, l0c_processing: bool = True, remove_l0a: bool = False, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0 processing of a specific DISDRODB station from the terminal.

Parameters:

data_source (str) – Institution name (when campaign data spans more than 1 country), or country (when all campaigns (or sensor networks) are inside a given country). Must be UPPER CASE.
campaign_name (str) – Campaign name. Must be UPPER CASE.
station_name (str) – Station name
l0a_processing (bool) – Whether to launch processing to generate L0A Apache Parquet file(s) from raw data. The default value is True.
l0b_processing (bool) – Whether to launch processing to generate L0B netCDF4 file(s) from L0A data. The default value is True.
l0b_processing – Whether to launch processing to generate L0C netCDF4 file(s) from L0B data. The default value is True.
l0c_processing (bool) – Whether to launch processing to generate L0C netCDF4 file(s) from L0C data. The default is True.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
remove_l0b (bool) – Whether to remove the L0B files after having produced L0C netCDF files. The default is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0A, it processes just the first 3 raw data files for each station. For L0B, it processes 100 rows sampled from 3 L0A files for each station. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0a(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0A processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process.
debugging_mode (bool) – If True, it processes just the first 3 raw data files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0a_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0A processing of a station by invoking the disdrodb_run_l0a_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data to process for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0b(data_sources=None, campaign_names=None, station_names=None, remove_l0a: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0B processing of DISDRODB stations.

This function allows to launch the processing of many DISDRODB stations with a single command. From the list of all available DISDRODB L0A stations, it runs the processing of the stations matching the provided data_sources, campaign_names and station_names.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0B, it processes 100 rows sampled from 3 L0A files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0b_station(data_source, campaign_name, station_name, remove_l0a: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0B processing of a station by invoking the disdrodb_run_l0b_station command in the terminal.

Parameters:

data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.
data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
remove_l0a (bool, optional) – Whether to keep the L0A files after generating L0B netCDF files. The default value is False.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.

disdrodb.routines.wrappers.run_l0c(data_sources=None, campaign_names=None, station_names=None, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0C processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
remove_l0b (bool) – Whether to remove the L0B files after having produced L0C netCDF files. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L1B, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l0c_station(data_source, campaign_name, station_name, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0C processing of a station by invoking the disdrodb_run_l0c_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
remove_l0b (bool, optional) – Whether to remove the L0B files after generating L0C netCDF files. The default value is False.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l1(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L1 processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L1B, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l1_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L1 processing of a station by invoking the disdrodb_run_l1_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l2e(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2E processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L2E, it processes just 3 L1 files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l2e_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2E processing of a station by invoking the disdrodb_run_l2e_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l2m(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2M processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L2MB, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.wrappers.run_l2m_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2M processing of a station by invoking the disdrodb_run_l2m_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

Module contents#

DISDRODB L0 software.

disdrodb.routines.create_summary(data_sources=None, campaign_names=None, station_names=None, parallel=False, temporal_resolution='1MIN', data_archive_dir=None, metadata_archive_dir=None)[source][source]#

Create summary figures and tables for a set of DISDRODB station.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.
temporal_resolution (str) – Temporal resolution of the summary. The default value is 1MIN.

disdrodb.routines.create_summary_station(data_source, campaign_name, station_name, parallel=False, temporal_resolution='1MIN', data_archive_dir=None)[source][source]#: Create summary figures and tables for a DISDRODB station.

disdrodb.routines.run_l0(data_sources=None, campaign_names=None, station_names=None, l0a_processing: bool = True, l0b_processing: bool = True, l0c_processing: bool = True, remove_l0a: bool = False, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0 processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
l0a_processing (bool) – Whether to launch processing to generate L0A Apache Parquet file(s) from raw data. The default value is True.
l0b_processing (bool) – Whether to launch processing to generate L0B netCDF4 file(s) from L0A data. The default value is True.
l0c_processing (bool) – Whether to launch processing to generate L0C netCDF4 file(s) from L0B data. The default value is True.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
remove_l0b (bool) – Whether to remove the L0B files after having produced all L0C netCDF files. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0A, it processes just the first 3 raw data files. For L0B, it processes 100 rows sampled from 3 L0A files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0_station(data_source, campaign_name, station_name, l0a_processing: bool = True, l0b_processing: bool = True, l0c_processing: bool = True, remove_l0a: bool = False, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0 processing of a specific DISDRODB station from the terminal.

Parameters:

data_source (str) – Institution name (when campaign data spans more than 1 country), or country (when all campaigns (or sensor networks) are inside a given country). Must be UPPER CASE.
campaign_name (str) – Campaign name. Must be UPPER CASE.
station_name (str) – Station name
l0a_processing (bool) – Whether to launch processing to generate L0A Apache Parquet file(s) from raw data. The default value is True.
l0b_processing (bool) – Whether to launch processing to generate L0B netCDF4 file(s) from L0A data. The default value is True.
l0b_processing – Whether to launch processing to generate L0C netCDF4 file(s) from L0B data. The default value is True.
l0c_processing (bool) – Whether to launch processing to generate L0C netCDF4 file(s) from L0C data. The default is True.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
remove_l0b (bool) – Whether to remove the L0B files after having produced L0C netCDF files. The default is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0A, it processes just the first 3 raw data files for each station. For L0B, it processes 100 rows sampled from 3 L0A files for each station. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0a(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0A processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process.
debugging_mode (bool) – If True, it processes just the first 3 raw data files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0a_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0A processing of a station by invoking the disdrodb_run_l0a_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data to process for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0b(data_sources=None, campaign_names=None, station_names=None, remove_l0a: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0B processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
remove_l0a (bool) – Whether to keep the L0A files after having generated the L0B netCDF products. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is True.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L0B, it processes 100 rows sampled from 3 L0A files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0b_station(data_source, campaign_name, station_name, remove_l0a: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0B processing of a station by invoking the disdrodb_run_l0b_station command in the terminal.

Parameters:

data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.
data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
remove_l0a (bool, optional) – Whether to keep the L0A files after generating L0B netCDF files. The default value is False.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.

disdrodb.routines.run_l0c(data_sources=None, campaign_names=None, station_names=None, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0C processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
remove_l0b (bool) – Whether to remove the L0B files after having produced L0C netCDF files. The default value is False.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L1B, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l0c_station(data_source, campaign_name, station_name, remove_l0b: bool = False, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L0C processing of a station by invoking the disdrodb_run_l0c_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
remove_l0b (bool, optional) – Whether to remove the L0B files after generating L0C netCDF files. The default value is False.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l1(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L1 processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L1B, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l1_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L1 processing of a station by invoking the disdrodb_run_l1_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l2e(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2E processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L2E, it processes just 3 L1 files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l2e_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2E processing of a station by invoking the disdrodb_run_l2e_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l2m(data_sources=None, campaign_names=None, station_names=None, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2M processing of DISDRODB stations.

Parameters:

data_sources (list) – Name of data source(s) to process. The name(s) must be UPPER CASE. If campaign_names and station are not specified, process all stations. The default value is None.
campaign_names (list) – Name of the campaign(s) to process. The name(s) must be UPPER CASE. The default value is None.
station_names (list) – Station names to process. The default value is None.
force (bool) – If True, overwrite existing data into destination directories. If False, raise an error if there are already data into destination directories. The default value is False.
verbose (bool) – Whether to print detailed processing information into terminal. The default value is False.
parallel (bool) – If True, the files are processed simultaneously in multiple processes. Each process will use a single thread to avoid issues with the HDF/netCDF library. By default, the number of process is defined with os.cpu_count(). If False, the files are processed sequentially in a single process. If False, multi-threading is automatically exploited to speed up I/0 tasks.
debugging_mode (bool) – If True, it reduces the amount of data to process. For L2MB, it processes just 3 L0B files. The default value is False.
data_archive_dir (str (optional)) – The directory path where the DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines.run_l2m_station(data_source, campaign_name, station_name, force: bool = False, verbose: bool = False, debugging_mode: bool = False, parallel: bool = True, data_archive_dir: str | None = None, metadata_archive_dir: str | None = None)[source][source]#

Run the L2M processing of a station by invoking the disdrodb_run_l2m_station command in the terminal.

Parameters:

data_source (str) – The name of the data source.
campaign_name (str) – The name of the campaign.
station_name (str) – The name of the station.
force (bool, optional) – If True, overwrite existing data in destination directories. The default value is False.
verbose (bool, optional) – If True, print detailed processing information to the terminal. The default value is False.
debugging_mode (bool, optional) – If True, reduce the amount of data processed for debugging. The default value is False.
parallel (bool, optional) – If True, process files in multiple processes simultaneously. The default value is True.
data_archive_dir – The directory path where the local DISDRODB Data Archive is located. The directory path must end with <...>/DISDRODB. If None, it uses the data_archive_dir path specified in the DISDRODB active configuration.
metadata_archive_dir – The directory path where the DISDRODB Metadata Archive is located. The directory path must end with <...>/DISDRODB-METADATA/DISDRODB. If None, it uses the metadata_archive_dir path specified in the DISDRODB active configuration.

disdrodb.routines package

Contents

disdrodb.routines package#

Submodules#

disdrodb.routines.l0 module#

disdrodb.routines.l1 module#

disdrodb.routines.l2 module#

disdrodb.routines.options module#

disdrodb.routines.wrappers module#

Module contents#