MainDimensionsHaver

class PlasmaCalcs.dimensions.main_dimensions.MainDimensionsHaver

Bases: DimensionHaver

All the dimensions which remain after indexing by all other DimensionHavers.

E.g., for simulation output from a 1024 x 512 x 512 simulation (in x, y, z),

for a given fluid, jfluid, component, and snapshot,

the array would be (1024, 512, 512).

This is probably the shape of (most) arrays as stored in memory.

use self.slices to control slicing behavior

Subclasses should implement:

maindims: listlike of strs

tells main dimensions. Will be accessed as instance.maindims.

Best to use a property if maindims depends on class.

_get_maindims_coords_base: method

self._get_maindims_coords_base() should return dict of {dim: coords},

for dims in self.maindims, in self.coords_units unit system and ignoring self.slices.

Methods

`__init_subclass__`(*[, dimension, dim_plural])	called when subclassing DimensionHaver; sets some useful attributes related to dimension.
`__setup_haver__`(dim_cls, **kw_super)	called by dim_cls.setup_haver(cls) when setting up cls so that it "has" dim_cls dimension.
`as_single_dimpoint`([values, dims])	return DimPoint with values for dims, but raise DimensionValueError if any value is_iterable_dim.
`assign_dim_coords`(array, *dims[, skip])	assign all dimensions in self as coords for array.
`assign_maindims_coords`(array)	assign maindims dims and coords, based on self.get_maindims_coords() with slices=None.
`check_pickle`([x])	checks that self (or, x, if provided) is pickleable, by pickling then unpickling.
`current_n_dimpoints`([dims])	return number of points represented by current values of dims.
`dim_values`([dims])	return dict of current values for dimensions in self.
`dims_apply`(funcname, *args_func[, dims])	apply funcname to each dimension in self, with args_func and kw_func.
`dims_get`(attr[, dims])	return dict of {dim: getattr(self.dimensions[dim], attr) for dim in dims}.
`enumerate_dimpoints`([dims, all])	iterate through values of dims, yielding (idx, DimPoint) pairs.
`get_behavior`([keys])	return value of self.behavior.
`get_first_dimpoint`([dims, enumerate])	return DimPoint taking the first value of each dim in self.dimensions.
`get_maindims_coord_xarrays`([dim])	return dict of {dim: xarray of coords for dim} for dim in self.maindims.
`get_maindims_coords`(*[, slices, coords_units])	return dict of {dim: coords} for all dims in self.maindims, applying self.slices appropriately.
`get_maindims_coords_idx`()	return dict of maindims coords' indices, based on self.slices.
`get_maindims_dgrid`(dim)	return grid spacing associated with dim values (dim in self.maindims).
`get_ncpu`()	returns ncpu, but if None, return multiprocessing.cpu_count() instead.
`iter_dimpoints`([dims, all, restore, enumerate])	iterate through values of dims, returning DimPoints and setting dim values during iteration.
`load_across_dims`(loader, *args_loader[, ...])	return loader(...), iterating & joining across each dimension.
`load_across_dims_implied_by`(var, loader, ...)	return loader(...), iterating & joining across each dimension implied by var.
`load_maindims_var`(var, *args[, u, assign_labels])	return var, formatted as an xarray with proper details for PlasmaCalcs.
`load_maindims_var_across_dims`(var[, dims, ...])	load maindims var across these dims.
`maintaining_attrs`(attrs, *attrs_as_flags)	returns context manager which restores attrs of self to their original values, upon exit.
`pop_dim_keys`(kw)	return ({key: kw.pop(key) for key in self.dimensions if key in kw}, kw).
`set_attrs`(**attrs)	sets these attrs in self.
`set_pop_dim_attrs`(kw)	set self.{key} = kw.pop(key) for each key in self.dimensions if key in kw.
`slice_maindims`(array, **kw_xarray_isel)	slice maindims of array using self.slices.
`slicestr`(*[, sep, keep_None])	string representation of self.slices, for use in filenames, titles, etc.
`standardized_slices`()	returns a copy of self.slices, but calling interprets_fractional_indexing on all slices,
`title_with_slices`([slices, sep, keep_None])	return self.title with slicestr appended (after sep), if slicestr is not empty.
`using_attrs`([attrs_as_dict, _unset_sentinel])	returns context manager which sets attrs of obj upon entry; restores original values upon exit.
`using_first_dimpoint`([dims])	return context manager which sets dimensions to their first values (when called); restore original on exit.
`zeros_like_maindims`(**kw_zeros_like)	return array of all 0s like maindims var results (use ints to save space).
`_apply_maindims_slices_to_dict`(d)	slice entries in dict using self.slices.
`_assert_is_maindim`(dim)	asserts dim in self.maindims; crash with DimensionKeyError if not.
`_call_loader0_at_dimpoint`(dimpoint, loader0, ...)	return self._call_loader_at_dimpoint(dimpoint, loader0, ...).
`_call_loader_at_dimpoint`(dimpoint, loader, ...)	with self.using(*dimpoint): return loader(args_loader, **kw_loader)
`_default_ncoarse`()	return default for ncoarse during self.load_across_dims.
`_default_ncpu`()	return default for ncpu during self.load_across_dims.
`_default_print_freq`()	returns default for print_freq.
`_default_timeout`()	return default for timeout during self.load_across_dims.
`_get_dimensions_dict`()	return dict of dimensions in self; {dimension name: Dimension object}
`_get_maindims_coords_base`()	return dict of {dim: coords} for all dimensions in self.maindims, ignoring self.slices.
`_get_maindims_coords_tiled`()	returns self.get_maindims_coords() with tile_snaps applied appropriately.
`_help_specialized_kw_call_options`()	returns dict of docstrings for specialized kw call options for self.
`_load_across_dims_postprocess`(arrs, dims, *, ...)	postprocessing results from load_across_dims, after all tasks have been completed.
`_load_across_dims_postprocess__arrs_to_nparr`(...)	returns single numpy array (with the data from arrs, which is an array of arrays).
`_load_across_dims_postprocess__broadcast_shapes`(...)	handles array broadcasting for _load_across_dims_postprocess.
`_load_maindims_var_with_labels`(var, args, *kw)	equivalent to self.load_maindims_var(var, args, assign_labels=True, *kw)
`_load_maindims_var_without_labels`(var, ...)	equivalent to self.load_maindims_var(var, args, assign_labels=False, *kw)
`_multi_slices_dims_default`()	function to get multi_slices dims, by default.
`_new_multi_slices`(**kw)	called when self.multi_slices accessed but not yet set; create & return new MultiSlices.
`_repr_contents`()	return list of contents to go in repr of self.
`_slice_maindims_numpy`(array, *[, h5])	slice first len(maindims) dims of array, using self.slices.
`_slices_which_scalarize`()	return list of maindims which become scalars when hit by self.slices.
`_special_dims_shifters`(dimnames, _shift_special)	return (dict for self.using(...), dict for result.isel(...)) to _shift_special_dims.
`_specialized_kw_call_options`(kw)	specialize popped kw_call_options, adjusting keys and/or values as needed,
`_tile_snaps_snaps`()	returns [pre-current, post-current] snaps to tile along based on self.tile_snaps and self.snap.

Attributes

`array_MBmax`	UNSET, None, or number
`behavior`	dict of {attr: self.attr} for attr in self.behavior_attrs.
`behavior_attrs`	list of attrs in self which control behavior of self.
`cls_behavior_attrs`
`dimensions`	dict of dimensions in self; {dimension name: Dimension object}.
`dims`	return dict of current values for dimensions in self.
`maindims`
`maindims_full_shape`	self.maindims_shape when self.slices=None
`maindims_full_size`	self.maindims_size when self.slices=None
`maindims_full_sizes`	self.maindims_sizes when self.slices=None
`maindims_means`	whether to immediately take means across maindims when loading arrays.
`maindims_shape`	tuple of (len(self.get_maindims_coords()[dim]) for dim in self.maindims).
`maindims_size`	product of terms in self.maindims_shape.
`maindims_sizes`	dict of {dim: size of dim} for dim in self.maindims.
`maintaining`	alias to maintaining_attrs
`multi_slices`	dict of {key: slices dict}.
`multi_slices_ikeep`	int or number between -1 < ikeep < 1
`multi_slices_ndim`	None or int
`ncoarse`	int
`ncpu`	None or int
`nondim_behavior_attrs`	list of attrs in self which control behavior of self, but which are NOT in self.dimensions.
`print_freq`	None, or number (possibly negative or 0)
`print_freq_explicit`	like self.print_freq, but converts UNSET to value based on self.verbose,
`slices`	slices for maindims when loading arrays & during get_maindims_coords.
`tile_snaps`	instructions for tiling results from snaps, concatenating along a maindim.
`timeout`	None or int
`using`	alias to using_attrs

classmethod __init_subclass__(*, dimension=None, dim_plural=None, **kw_super)

called when subclassing DimensionHaver; sets some useful attributes related to dimension.

dimension: str or None

name of dimension associated with this subclass.

if None, no particular dimension associated with this subclass.

dim_plural: str or None

plural form of dimension. if None, use str(dimension)+’s’.

Sets various attributes in cls:

cls._dimension = dimension,

cls._dim_plural = dim_plural,

cls._dim_types = dict of all {dimension name: Dimension subclass} from cls and cls.__bases__

(note - connecting self._dimension to this dict is not handled here;

it is handled by __setup_haver__ called by Dimension.setup_haver)

classmethod __setup_haver__(dim_cls, **kw_super): called by dim_cls.setup_haver(cls) when setting up cls so that it “has” dim_cls dimension.

_apply_maindims_slices_to_dict(d): slice entries in dict using self.slices. returns d. (Note: d will be directly altered.)

_assert_is_maindim(dim): asserts dim in self.maindims; crash with DimensionKeyError if not.

_call_loader0_at_dimpoint(dimpoint, loader0, args_loader, *, nload, MBmax, **kw_loader): return self._call_loader_at_dimpoint(dimpoint, loader0, …).

Also does memory size check based on nload and MBmax.

intended for internal use only, inside load_across_dims.

arr0 is handled differently from other arrays in order to do some checks.

_call_loader_at_dimpoint(dimpoint, loader, args_loader, **kw_loader): with self.using(**dimpoint): return loader(*args_loader, **kw_loader)

_default_ncoarse(): return default for ncoarse during self.load_across_dims. returns DEFAULTS.LOADING_NCOARSE

_default_ncpu(): return default for ncpu during self.load_across_dims. returns DEFAULTS.LOADING_NCPU

_default_print_freq(): returns default for print_freq. Here, returns UNSET; i.e., infer from self.verbose.

(See self.print_freq_explicit to get the actual value of print_freq.)

_default_timeout(): return default for timeout during self.load_across_dims. returns DEFAULTS.LOADING_TIMEOUT

_get_dimensions_dict(): return dict of dimensions in self; {dimension name: Dimension object}

_get_maindims_coords_base(): return dict of {dim: coords} for all dimensions in self.maindims, ignoring self.slices.

E.g., {‘x’: xcoords, ‘y’: ycoords}, where xcoords and ycoords are numpy arrays.

Results should be in self.coords_units (default: self.units) unit system.

See also: self.get_maindims_coords().

_get_maindims_coords_tiled(): returns self.get_maindims_coords() with tile_snaps applied appropriately.

(self.get_maindims_coords() calls this function internally when self.tile_snaps != False).

_help_specialized_kw_call_options(): returns dict of docstrings for specialized kw call options for self.

The implementation here just returns an empty dict, but subclass may override.

_load_across_dims_postprocess(arrs, dims, *, it_dimnames, assign_coords, isel=None): postprocessing results from load_across_dims, after all tasks have been completed.

i.e., after all arrays have been loaded, join them all together.

_load_across_dims_postprocess__arrs_to_nparr(arr0, arrs): returns single numpy array (with the data from arrs, which is an array of arrays).

_load_across_dims_postprocess__broadcast_shapes(arr0, arrs, it_dimnames): handles array broadcasting for _load_across_dims_postprocess.

returns (arr0, arrs), but with all arrs having the same shape as arr0. Crash if not possible.

CAUTION: may edit arrs in-place.

(Won’t edit individual arrays’ data; just arrs which contains the arrays.)

NOTE: arr0 in result isn’t necessarily arrs.flat[0].

arr0 in result is the “model array”, used to model coords & dims on.

arr0 = first DataArray with size == max(size) across all arrays.

E.g. if array shapes are [(), (), (7,1), (1,3), (7,3), (7,3) ()],

then use arr0 = the first DataArray with shape (7,3).

if neither array with shape (7,3) is a DataArray (i.e., has coords…)

then crash instead (because we can’t properly assign coords to result.)

it_dimnames: list of names of iterable dimensions; sometimes included in error message.

_load_maindims_var_with_labels(var, *args, **kw): equivalent to self.load_maindims_var(var, *args, assign_labels=True, **kw)

_load_maindims_var_without_labels(var, *args, **kw): equivalent to self.load_maindims_var(var, *args, assign_labels=False, **kw)

_multi_slices_cls: alias of MultiSlices

_multi_slices_dims_default(): function to get multi_slices dims, by default.

Would love to use (lambda: self.maindims) instead of this function,

however lambdas are not pickleable, thus incompatible with multiprocessing module.

Providing this function allows the default multi_slices to still be pickleable.

_new_multi_slices(**kw): called when self.multi_slices accessed but not yet set; create & return new MultiSlices.

_repr_contents(): return list of contents to go in repr of self.

_slice_maindims_numpy(array, *, h5=False)

slice first len(maindims) dims of array, using self.slices.

h5: bool

whether ‘array’ might be an h5py dataset.

if True, handle negative step in the intuitive way;

i.e. slice with positive step, to select the expected points, then reverse order.

(This is necessary because h5 datasets crash when sliced by negative step…)

_slices_which_scalarize(): return list of maindims which become scalars when hit by self.slices.

_special_dims_shifters(dimnames, _shift_special): return (dict for self.using(…), dict for result.isel(…)) to _shift_special_dims.

See docs of load_across_dims for more details.

_specialized_kw_call_options(kw): specialize popped kw_call_options, adjusting keys and/or values as needed,

to be suitable to pass to self.using(**kw).

kw may be edited IN PLACE.

Overriding this is discouraged, unless using property setters/getters is truly insufficient.

If overriding this method, consider also overriding self._help_specialized_kw_call_options,

to add documentation (inside self.help_call_options()) for any specialized kw call options.

self.__call__ uses this method as follows:

using=self._pop_kw_call_options(kw)

using = self._specialize_using_kw_call_options(using)

with self.using(**using):

# <– majority of self.__call__ functionality goes here.

The implementation here looks for any maindims in kw, using them to adjust relevant slices.

Example: kw=dict(z=slice(100)) –> returns dict(slices={**self.slices, **{‘z’: slice(100)}})

Example: kw=dict(y=9, slices=dict(x=8)) –> returns dict(slices=dict(x=8, y=9)).

All other kwargs remain unchanged.

_tile_snaps_snaps(): returns [pre-current, post-current] snaps to tile along based on self.tile_snaps and self.snap.

(raise ValueError if self._tile_snaps is False, InputError if not exactly 1 ‘current’ snap indicator,

or SnapValueError if self.snap_is_iterable())

pre-current and post-current are (possibly empty) lists of Snap objects.

property array_MBmax: UNSET, None, or number

maximum result size allowed, in Megabytes.

will raise a MemorySizeError if result size would be larger than this.

UNSET –> use DEFAULTS.ARRAY_MBYTES_MAX (default: 1000 MB).

None –> no limit.

Assumes that each result (at each dimpoint) will be the same size.

as_single_dimpoint(values=None, *, dims=None, **values_as_kw): return DimPoint with values for dims, but raise DimensionValueError if any value is_iterable_dim.

values: None or dict

values to use for the dimpoint.

values will be joined with **values_as_kw; provided any of either will be equivalent.

E.g. can use values={‘fluid’: ‘e’} or use fluid=’e’.

if any are provided –> use values corresponding to self.{dim}=values[dim] for dim in dims.

else –> use values of self.{dim} for dim in dims. (equivalent: self.dims_apply(‘_as_single’, dims=dims))

dims: None or iterable of strs appearing in self.dimensions.keys()

dimensions to include.

None –> infer dimensions from keys of values (and values_as_kw).

if no values were provided (values=None, and empty values_as_kw),

use all dimensions from self.dimensions.keys().

additional kwargs provide other {dim: value} items.

Examples:

self.as_single_dimpoint() –> DimPoint({dim: self.{dim} for dim in self.dimensions})

self.as_single_dimpoint({‘fluid’: ‘e’}) –> DimPoint({‘fluid’: ‘e’})

self.as_single_dimpoint(fluid=’e’) –> DimPoint({‘fluid’: ‘e’})

self.as_single_dimpoint({‘fluid’: ‘e’}, snap=0) –> DimPoint({‘fluid’: ‘e’, ‘snap’: 0})

self.as_single_dimpoint(dims=[‘fluid’, ‘snap’]) –> DimPoint({‘fluid’: self.fluid, ‘snap’: self.snap})

assign_dim_coords(array, *dims, skip=[]): assign all dimensions in self as coords for array. (self.assign_{dim}_coord(array))

Assumes array is an xarray and does not have any dimensions in self.

(array is not edited directly; returns result of assigning coords.)

dims: iterable of dimensions in self

assign only these dimensions as coords. (use all dimensions if len(dims)==0)

skip: iterable of dimensions in self

do not assign these dimensions as coords.

assign_maindims_coords(array): assign maindims dims and coords, based on self.get_maindims_coords() with slices=None.

array must have same shape as implied by maindims and coords.

if array is 0D, just return a 0D xr.DataArray.

returns an xarray with proper details for PlasmaCalcs.

(if self._slice_maindims_in_load_direct, actually uses self.slices instead of slices=None.)

This function creates a *new* xarray based on array, and maindims & coords are >0 dimensional.

This is not like assign_{dim}_coord functions, which assign 0D coord to an existing xarray.

property behavior: dict of {attr: self.attr} for attr in self.behavior_attrs. Note dims are separate;

dims go in behavior.dims. E.g. Behavior({‘units’:’si’,…}, dims={‘snap’:0,…}).

property behavior_attrs: list of attrs in self which control behavior of self.

Here, returns self.cls_behavior_attrs.

Subclasses could override if any behavior attrs are not known at the class-level,

e.g. if MySubclass’s list of behavior attrs varies between instances of MySubclass.

check_pickle(x=None): checks that self (or, x, if provided) is pickleable, by pickling then unpickling.

Returns result of unpickling. Useful for debugging.

current_n_dimpoints(dims=None): return number of points represented by current values of dims.

dims: None or iterable of strs appearing in self.dimensions.keys()

dimensions to consider. None –> use all dimensions.

E.g. current_n_dimpoints(self, dims=[‘fluid’, ‘snap’]) –> number of (fluid, snap) points;

e.g. 3 fluids and 2 snaps –> 6 points.

Note, for classes using maindims, maindims are not included in the number of dimpoints.

Equivalent to len(list(self.iter_dimpoints(dims=dims, current=True)))

dim_values(dims=None): return dict of current values for dimensions in self.

dims: None or iterable

if provided, only include these dimensions.

Equivalent: DimRegion(self.dims_get(‘v’, dims=dims))

property dimensions: dict of dimensions in self; {dimension name: Dimension object}.

e.g. {‘fluid’: self.fluid_dim, ‘snap’: self.snap_dim, …}.

property dims: return dict of current values for dimensions in self. Equivalent: self.dim_values()

dims_apply(funcname, *args_func, dims=None, **kw_func): apply funcname to each dimension in self, with args_func and kw_func.

dims: None or iterable of strs

if provided, only apply to these dimensions.

See also: dims_get

dims_get(attr, dims=None): return dict of {dim: getattr(self.dimensions[dim], attr) for dim in dims}.

dims: None or iterable

if provided, only include these dimensions.

See also: dims_apply

enumerate_dimpoints(dims=None, *, all=False): iterate through values of dims, yielding (idx, DimPoint) pairs.

idx is a dict of {dim: i} such that DimPoint values are {dim: dims[i] for dim,i in idx.items()}.

Also, during iteration, set self.{dim} = value, as with self.iter_dim.

Equivalent to self.iter_dimpoints(dims=dims, all=all, enumerate=True)

get_behavior(keys=None): return value of self.behavior.

keys: None or iterable

if provided, only include these attrs.

from nondim_behavior_attrs, or dims.

get_first_dimpoint(dims=None, *, enumerate=False): return DimPoint taking the first value of each dim in self.dimensions.

dims: None or iterable of strs appearing in self.dimensions.keys()

dimensions to include. None –> use all dimensions.

enumerate: bool

whether to return (idx, DimPoint) instead of just DimPoint.

get_maindims_coord_xarrays(dim=None): return dict of {dim: xarray of coords for dim} for dim in self.maindims.

x: None or str; if provided, return result[x] instead.

Result’s values use self.units; coords use self.coords_units (default==self.units).

(e.g., units=’cgs’, coord_units=’si’ –> values are in cgs, but coords are in SI.)

get_maindims_coords(*, slices=UNSET, coords_units=UNSET): return dict of {dim: coords} for all dims in self.maindims, applying self.slices appropriately.

E.g., {‘x’: xcoords, ‘y’: ycoords}, where xcoords and ycoords are numpy arrays.

Results will be in self.coords_units (default: self.units) unit system.

slices: UNSET, None, or dict

if provided (i.e. not UNSET) use these slices instead of self.slices.

equivalent to: with self.using(slices=slices): get_maindims_coords().

E.g., slices = None (or dict()) –> full maindims coords, without any slicing.

coords_units: UNSET, None, or str

if provided (i.e. not UNSET) use coords_units = coords_units instead of self.coords_units.

equivalent to: with self.using(coords_units=coords_units): get_maindims_coords().

E.g., coords_units = ‘si’ –> maindims coords in ‘si’ regardless of self.coords_units.

get_maindims_coords_idx(): return dict of maindims coords’ indices, based on self.slices.

E.g. if self.slices[‘x’] = slice(10, None, 4),

then result[‘x’] = array([10, 14, 18, …]).

Values in result are xarray.DataArrays,

with dim = the maindim, coords = the maindim’s coord values.

get_maindims_dgrid(dim): return grid spacing associated with dim values (dim in self.maindims).

Result is a 1D xarray.DataArray, with coords according to self.get_maindims_coords()[dim].

Result’s values use self.units; coords use self.coords_units (default==self.units).

(e.g., units=’cgs’, coord_units=’si’ –> values are in cgs, but coords are in SI.)

If dim’s gradient cannot be inferred, result is NaN.

Caution: this is the “underlying grid size”, regardless of self.slices.

E.g., slicing with slice(None,None,4) will NOT lead to 4 times larger result.

get_ncpu(): returns ncpu, but if None, return multiprocessing.cpu_count() instead.

(This is for convenience; using None will also work with any methods defined here.)

iter_dimpoints(dims=None, *, all=False, restore=True, enumerate=False): iterate through values of dims, returning DimPoints and setting dim values during iteration.

DimPoints are dicts of {dim: value} for dim in dims, where not is_iterable_dim(value).

Also, during iteration, set self.{dim} = value, as with self.iter_dim.

dims: None or iterable of strs appearing in self.dimensions.keys()

dimensions to consider. None –> use all dimensions.

all: bool

whether to iterate through all possible values, or only the current values.

False –> iterate through current values (e.g., self.snap, self.fluid, …).

similar to itertools.product(self.iter_snap(), self.iter_fluid(), …)

True –> iterate through all possible values (e.g., self.snaps, self.fluid, …)

similar to itertools.product(self.iter_snaps(), self.iter_fluids(), …)

Equivalent to all=False if all dims are set to None, e.g. self.snap=None, …

restore: bool

whether to restore original dim values after iteration.

enumerate: bool, default False

whether to yield indices too, i.e. (idx, DimPoint) instead of just DimPoint.

idx would be a dict of {dim: i} such that DimPoint values are {dim: dims[i] for dim,i in idx.items()}.

load_across_dims(loader, *args_loader, dims=[], assign_coords=None, loader0=None, _shift_special={}, **kw_loader)

return loader(…), iterating & joining across each dimension.

loader: callable of (*args_loader, **kw_loader) -> xarray.DataArray.

will call loader to get result values at each combination of dims values in self.

(loader will probably depend on dims values from self.)

dims: iterable of strs or Dimension objects

load across these Dimensions.
loads across the current values (when this method was called) of each dimension,
not necessarily “all” values. (e.g., self.snap, not self.snaps)
str values –> use self.dimensions[d] (where d is a str in dims).
len(dims)==0 –> just return loader(var, *args_loader, **kw_loader).
While loading, set dim.loading=True for each dim.

assign_coords: None or bool, default None

whether to dim.assign_coord for each result of loader, for each dimension.

None –> assign coord only if dim.name not already in array.coords.

loader0: None or callable

if provided, use loader0 to get the first array, then use loader for the rest.
Internally the first array’s .coords and .attrs are used to label the result;
however all other arrays do not need to be converted to xarray.

_shift_special: UNSET or dict of (dimstr: list of special values)

workaround to encourage loader0 to be called on a “usual” case, not a special case.

if provided, and dimstr in dims, and d=self.dimensions[dimstr] has multiple values,

with special_value first, and at least one non-special value later, then

internally rearrange dim values order before loading,

then rearrange result back to original order (via indexing).

E.g. _shift_special=dict(snap=[INPUT_SNAP]) –> apply loader0 to the first non-INPUT_SNAP,

if there are any non-INPUT_SNAP snap values in snap, and ‘snap’ in dims.

— MULTIPROCESSING STRATEGY OPTIONS (from self) —

timeout: None or int

max duration, in seconds. Must be None or integer (due to limitations of signal.alarm method)

None –> no time limit.

Note: if time_limit is reached, will raise a TimeoutError and save the result so far.

(in this case, any not-yet-calculated values will each be RESULT_MISSING.)

# [TODO] make this happen, without making self un-picklable:

in case of crash, results so far can be found in self._latest_load_tasks.

Then possibly continued via:

results = self._latest_load_tasks(…, reset=False, skip_done=True)
result = self._load_across_dims_postprocess(results, dims, …)
# [TODO] if crashing and resuming is common, make that easier to do^

if self.timeout has not been set, use DEFAULTS.LOADING_TIMEOUT (default: None).

ncpu: None or int

max number of cpus to use for multiprocessing.

None –> use multiprocessing.cpu_count()

int –> use this value. if 0 or 1, do not use multiprocessing here.

Note: will actually use min(ncpu, number of calls to be made);

e.g. if ncpu=4 but len(arg_kw_tuples)=2, will only use 2 cpus.

if self.ncpu has not been set, use DEFAULTS.LOADING_NCPU (default: 1).

ncoarse: int

if >1, group tasks into groups of size ncoarse before performing them.

if self.ncoarse has not been set, use DEFAULTS.LOADING_NCOARSE (default: 1).

print_freq: None, or number (possibly negative or 0)

>0 –> Minimum number of seconds between progress updates.
=0 –> print every progress update.
<0 –> never print progress updates.
None –> use DEFAULTS.PROGRESS_UPDATES_PRINT_FREQ

if self.print_freq has not been set, infer from self.verbose if it exists,

else use DEFAULTS.PROGRESS_UPDATES_PRINT_FREQ (default: 2).

additional args & kwargs are passed as loader(*args_loader, **kw_loader).

load_across_dims_implied_by(var, loader, *args_loader, assign_coords=None, _min_split=1, **kw_loader): return loader(…), iterating & joining across each dimension implied by var.

Equivalent to self.load_across_dims(loader, …, dims=self.match_var_loading_dims(var)).

var: str

variable which implies dims to load across, via self.match_var_loading_dims(var).

loader: callable of (*args_loader, **kw_loader) -> xarray.DataArray.

will call loader to get result values at each combination of dims values in self.

(loader will probably depend on dims values from self.)

assign_coords: None or bool, default None

whether to dim.assign_coord for each result of loader, for each dimension.

None –> assign coord only if dim.name not already in array.coords.

_min_split: int, default 1

if an implied dim has current_n() < min_split, don’t load across it.

1 –> no minimum.

additional args & kwargs are passed as loader(*args_loader, **kw_loader).

load_maindims_var(var, *args, u=None, assign_labels=True, **kw): return var, formatted as an xarray with proper details for PlasmaCalcs.

loading var should give an array with self.maindims as dimensions.

Also does these steps:

1) assign maindims coords via self.assign_maindims_coords().

2) slice array via self.slices.

3) convert units, if u is not None

4) set result.attrs[‘units’] = self.units

5) if self.maindims_means: take mean of result, across all maindims.

6) use result = self._maindims_postprocess_callback(result), if possible.

(Before those steps, first checks special cases:

- if tile_snaps, load for each snap then concatenate results appropriately.

- if using multi_slices, load for each slice then combine results into xarray.Dataset

(or combine into dict instead if assign_labels=False).)

u: None, value, or str

units factor for the result.

None –> don’t do any units conversions.

str –> multiply result by self.u(u)

value –> multiply result by u

assign_labels: bool

whether to assign_maindims_coords and self.record_units.

Recommend to always use True, unless using this function internally.

(e.g. for load_maindims_var_across_dims, only use the first time, for efficiency.)

IGNORED if self.maindims_means.

Note:

If load_direct(var) uses an override or gets from cache or self.setvars,

skip steps 1,2,3,4

([TODO] Might need to reconsider this behavior?)

load_maindims_var_across_dims(var, dims=None, *, skip=[], u=None, **kw): load maindims var across these dims. Use all dims from self.dimensions if dims is None.

Only loads across the current value of these dims (e.g., self.fluid, not self.fluids).

(Can set current value to multiple values e.g. self.component = (‘x’, ‘y’).)

u: None, value, or str

units factor for the result.

None –> don’t do any units conversions.

str –> multiply result by self.u(u)

value –> multiply result by u

property maindims_full_shape: self.maindims_shape when self.slices=None

property maindims_full_size: self.maindims_size when self.slices=None

property maindims_full_sizes: self.maindims_sizes when self.slices=None

property maindims_means: whether to immediately take means across maindims when loading arrays. (default False.)

True –> treat data across maindims as if it were the mean values, only.

Caution: this is different from taking means after doing calculations;

e.g., with maindims_means = True, ‘n*T’ –> mean(n)*mean(T), not mean(n*T).

property maindims_shape: tuple of (len(self.get_maindims_coords()[dim]) for dim in self.maindims).

Note, this should be sensitive to changes in self.slices. See also: self.maindims_full_shape.

property maindims_size: product of terms in self.maindims_shape.

Note, this should be sensitive to changes in self.slices. See also: self.maindims_full_size.

property maindims_sizes: dict of {dim: size of dim} for dim in self.maindims.

Note, this should be sensitive to changes in self.slices. See also: self.maindims_full_sizes.

property maintaining: alias to maintaining_attrs

maintaining_attrs(*attrs, **attrs_as_flags): returns context manager which restores attrs of self to their original values, upon exit.

E.g. maintaining_attrs(obj, ‘attr1’, ‘attr2’, attr3=True, attr4=False)

–> will restore upon exit, original values of obj.attr1, attr2, and attr3, but not attr4.

property multi_slices: dict of {key: slices dict}.

When getting any vars across maindims, make a Dataset by applying each of these, separately.

If len(multi_slices)>0 then ignore self.slices.

Can also provide special keys ‘ndim’ and/or ‘ikeep’ to create special slices:

Example: if self.maindims=[‘x’, ‘y’, ‘z’], then self.multi_slices = dict(ndim=2, ikeep=0)

is equivalent to: self.multi_slices = dict(x_y=dict(z=0), x_z=dict(y=0), y_z=dict(x=0))

Details:

ndim: None or int

None –> ignore, and do not create special slices.

int –> create special slices to keep this many dims after applying each slice.

Example: MultiSlices(ndim=2) is shorthand for

“MultiSlices with one slices for every possible combination of keeping 2 dims”.

Example: MultiSlices(ndim=2, dims=[‘x’, ‘y’, ‘z’], ikeep=0) is equivalent to:

MultiSlices(keep_x_y=dict(z=0), keep_y_z=dict(x=0), keep_x_z=dict(y=0))

Example: MultiSlices(ndim=1, dims=[‘x’, ‘y’, ‘z’], ikeep=0) is equivalent to:

MultiSlices(keep_x=dict(y=0, z=0), keep_y=dict(x=0, z=0), keep_z=dict(x=0, y=0))

ikeep: int or number between -1 < ikeep < 1

index to take when picking a single value for sliced dimensions for special slices.

Default is 0, e.g. when slicing x, keep x[0].

int –> when slicing dim, keep dim[ikeep]. E.g. 10 –> keep x[10]

non-int between -1 and 1 –> multiply by length of dim to get index.

see interprets_fractional_indexing for more details.

Can also set these as attributes of self.multi_slices to achieve the same effect.

E.g. self.multi_slices.ndim = 2

property multi_slices_ikeep: int or number between -1 < ikeep < 1

index to take when picking a single value for sliced dimensions for special slices.

Default is 0, e.g. when slicing x, keep x[0].

int –> when slicing dim, keep dim[ikeep]. E.g. 10 –> keep x[10]

non-int between -1 and 1 –> multiply by length of dim to get index.

see interprets_fractional_indexing for more details.

property multi_slices_ndim: None or int

None –> ignore, and do not create special slices.

int –> create special slices to keep this many dims after applying each slice.

Example: MultiSlices(ndim=2) is shorthand for

“MultiSlices with one slices for every possible combination of keeping 2 dims”.

Example: MultiSlices(ndim=2, dims=[‘x’, ‘y’, ‘z’], ikeep=0) is equivalent to:

MultiSlices(keep_x_y=dict(z=0), keep_y_z=dict(x=0), keep_x_z=dict(y=0))

Example: MultiSlices(ndim=1, dims=[‘x’, ‘y’, ‘z’], ikeep=0) is equivalent to:

MultiSlices(keep_x=dict(y=0, z=0), keep_y=dict(x=0, z=0), keep_z=dict(x=0, y=0))

property ncoarse: int

if >1, group tasks into groups of size ncoarse before performing them.

property ncpu: None or int

max number of cpus to use for multiprocessing.

None –> use multiprocessing.cpu_count()

int –> use this value. if 0 or 1, do not use multiprocessing here.

Note: will actually use min(ncpu, number of calls to be made);

e.g. if ncpu=4 but len(arg_kw_tuples)=2, will only use 2 cpus.

see also: self.get_ncpu() to read actual number of cpus when self.ncpu is None.

property nondim_behavior_attrs: list of attrs in self which control behavior of self, but which are NOT in self.dimensions.

pop_dim_keys(kw): return ({key: kw.pop(key) for key in self.dimensions if key in kw}, kw).

property print_freq: None, or number (possibly negative or 0)

>0 –> Minimum number of seconds between progress updates.

=0 –> print every progress update.

<0 –> never print progress updates.

None –> use DEFAULTS.PROGRESS_UPDATES_PRINT_FREQ

property print_freq_explicit: like self.print_freq, but converts UNSET to value based on self.verbose,

UNSET –> result depends on self.verbose:

False or <=0 –> -1

True or (>=1 and <5) –> None

>=5 –> 0 (i.e. print every progress update)

if self.verbose doesn’t exist –> None

if result would be None, instead give DEFAULTS.PROGRESS_UPDATES_PRINT_FREQ.

set_attrs(**attrs): sets these attrs in self.

set_pop_dim_attrs(kw): set self.{key} = kw.pop(key) for each key in self.dimensions if key in kw.

slice_maindims(array, **kw_xarray_isel): slice maindims of array using self.slices. See help(type(self).slices) for more details.

(if slices is an empty dict, return array, unchanged, without making a copy.)

Only slice dims which actually appear in array.

property slices: slices for maindims when loading arrays & during get_maindims_coords.

E.g. slices = dict(x=slice(0,50), y=7)

–> slice arrays along x & y, taking the first 50 x values, and only the 7th y value.

Setting self.slices = None means “no slices” and will actually set self.slices to an empty dict().

(self.slices = None –> (self.slices is None) == False, but (self.slices == dict()) == True.)

Notes:

- only applies slices along arrays which actually contain the related coordinates,

e.g. if z=10 appears in slice but loading an array with only x & y, won’t apply z=10 slice.

- supports fractional indexing, as per interprets_fractional_indexing.

Non-integer values between -1 and 1 can be used to infer to a fraction of the dimension length,

with negative values referring to a distance from the end, just like with integer indexing.

Example: dict(x=slice(-0.3, None, 0.01), y=0.8), where x and y each have length 1000

–> equivalent to dict(x=slice(-300, None, 10), y=800).

slicestr(*, sep=', ', keep_None=False): string representation of self.slices, for use in filenames, titles, etc.

comma-separated, alphabetized, ignoring slice(None).

Supports single-indexes (e.g. x=5), slices (e.g. y=slice(0, 4)),

and fractional indexing (e.g. z=slice(0, 0.5, 0.01)),

though fractional indexing will be converted to ints.

sep: str, separator between slices

keep_None: bool, whether to keep slices with value None in the string.

standardized_slices(): returns a copy of self.slices, but calling interprets_fractional_indexing on all slices,

using lengths from self.maindims_full_sizes.

property tile_snaps: instructions for tiling results from snaps, concatenating along a maindim.

False –> no tiling.

(dim, snaps iterable) –> tile snaps along dim, concatenating along dim.

Exactly 1 snap indicator must be None, ‘here’, or ‘current’ to indicate “the current snap”.

E.g., (‘x’, [5, ‘current’, ‘snap100’]) –> results get stacked along x, like this:

[results from 5th snap, results from current snap, results from snap with name==’snap100’]

Coordinates will be shifted by appropriate offsets, too,

with coords in ‘current’ matching maindims coords for current snap.

property timeout: None or int

max duration, in seconds. Must be None or integer (due to limitations of signal.alarm method)

None –> no time limit.

Note: if time_limit is reached, will raise a TimeoutError and save the result so far.

(in this case, any not-yet-calculated values will each be RESULT_MISSING.)

title_with_slices(slices=UNSET, *, sep=', ', keep_None=False): return self.title with slicestr appended (after sep), if slicestr is not empty.

see self.slicestr() for more details.

slices: None, dict, or UNSET

slices to use instead of self.slices, if provided (i.e. not UNSET).

(original slices will be restored after this operation)

property using: alias to using_attrs

using_attrs(attrs_as_dict={}, _unset_sentinel=ATTR_UNSET, **attrs_and_values): returns context manager which sets attrs of obj upon entry; restores original values upon exit.

_unset_sentinel: any value, default ATTR_UNSET

upon entry, delete any attrs with value _unset_sentinel (compared via ‘is’).

E.g. using_attrs(obj, _unset_sentinel=None, x=None) –> del obj.x upon entry.

using_first_dimpoint(dims=None): return context manager which sets dimensions to their first values (when called); restore original on exit.

Useful for testing a single code at a single dimpoint without needing to set each dimension individually.

dims: None or iterable of strs appearing in self.dimensions.keys()

dimensions to include. None –> use all dimensions.

zeros_like_maindims(**kw_zeros_like): return array of all 0s like maindims var results (use ints to save space).

Most users will find it sufficient to just load self.maindims_coords(),

but zeros_like_maindims() is provided for more advanced abstractions,

which rely on the fully-formed array, without caring about the actual values.

(see, e.g., LoiLoader’s get_dloi().)

Just like self.get_maindims_coords(), the result here depends on self.slices.

kwargs go to np.zeros(). Default dtype is int, to save space.