functions module

Module containing most of the utility functions for the binarycpython package

Functions here are mostly functions used in other classes/functions, or useful functions for the user

Tasks:
  • TODO: change all prints to verbose_prints

binarycpython.utils.functions.ANSI_colours()[source]
class binarycpython.utils.functions.AutoVivificationDict[source]

Bases: dict

Implementation of perl’s autovivification feature, by overriding the get item and the __iadd__ operator (https://docs.python.org/3/reference/datamodel.html?highlight=iadd#object.__iadd__)

This allows to set values within a subdict that might not exist yet:

Example

newdict = {} newdict[‘example’][‘mass’] += 10 print(newdict) >>> {‘example’: {‘mass’: 10}}

class binarycpython.utils.functions.BinaryCEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Bases: json.encoder.JSONEncoder

Encoding class function to attempt to convert things to strings.

default(o)[source]

Converting function. Well, could be more precise. look at the JSON module

class binarycpython.utils.functions.Capturing(iterable=(), /)[source]

Bases: list

Context manager to capture output and store it

__enter__()[source]

On entry we capture the stdout output

__exit__(*args)[source]

On exit we release the capture again

binarycpython.utils.functions.bin_data(value, binwidth)[source]

Function that bins the data

Uses the absolute value of binwidth

class binarycpython.utils.functions.binarycDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)[source]

Bases: json.decoder.JSONDecoder

Custom decoder to transform the numbers that are strings to actual floats

decode(s)[source]

Entry point function for decoding

binarycpython.utils.functions.binaryc_json_serializer(obj)[source]

Custom serialiser for binary_c to use when functions are present in the dictionary that we want to export.

Function objects will be turned into str representations of themselves

Parameters

obj (Any) – The object that might not be serialisable

Return type

Any

Returns

Either string representation of object if the object is a function, or the object itself

binarycpython.utils.functions.call_binary_c_config(argument)[source]

Function to interface with the binary_c config file

input:
  • argument: argument for the binary_c config

Returns

  • raw output of binary_c-config

class binarycpython.utils.functions.catchtime[source]

Bases: object

Context manager to calculate time spent

__enter__()[source]

On entry we start the clock

__exit__(type, value, traceback)[source]

On exit we stop the clock and measure the time spent

binarycpython.utils.functions.check_if_in_shell()[source]

Function to check whether the script is running from a shell

binarycpython.utils.functions.conv_time_units(t)[source]

Converts time (t, in seconds, passing in as the only argument) to seconds, minutes or hours depending on its magnitude. Returns a tuple (t,units).

binarycpython.utils.functions.convert_bytes(size)[source]

Function to return the size + a magnitude string

binarycpython.utils.functions.convfloat(x)[source]

Convert scalar x to a float if we can, in which case return the float, otherwise just return x without changing it. Usually, x is a string, but could be anything that float() can handle without failure.

binarycpython.utils.functions.count_keys_recursive(input_dict)[source]

Function to count the total number of keys in a dictionary

binarycpython.utils.functions.create_arg_string(arg_dict, sort=False, filter_values=False)[source]

Function that creates the arg string for binary_c. Takes a dictionary containing the arguments and writes them to a string This string is missing the ‘binary_c ‘ at the start.

Parameters
  • arg_dict (dict) – dictionary

  • sort (bool) – (optional, default = False) Boolean whether to sort the order of the keys.

  • filter_values (bool) – (optional, default = False) filters the input dict on keys that have NULL or function as value.

Return type

str

Returns

The string built up by combining all the key + value’s.

binarycpython.utils.functions.create_hdf5(data_dir, name)[source]
Function to create an hdf5 file from the contents of a directory:
  • settings file is selected by checking on files ending on settings

  • data files are selected by checking on files ending with .dat

TODO: fix missing settings files

Parameters
  • data_dir (str) – directory containing the data files and settings file

  • name (str) – name of hdf5file.

Return type

None

binarycpython.utils.functions.custom_sort_dict(input_dict)[source]

Returns a dictionary that is ordered, but can handle numbers better than normal OrderedDict

When the keys of the current dictionary are of mixed type, we first find all the unique types. Sort that list of type names. Then find the values that fit that type. Sort those and append them to the sorted keys list. This is done until all the keys are sorted.

All objects other than dictionary types are directly return as they are

binarycpython.utils.functions.datalinedict(line, parameters)[source]

Convert a line of data to a more convenient dictionary. :param line = a line of data as a string: :param parameters = a list of the parameter names:

Note: if the parameter is a floating point number, it will be converted to Python’s float type.

binarycpython.utils.functions.example_parse_output(output, selected_header)[source]

Function that parses output of binary_c. This version serves as an example and is quite detailed. Custom functions can be easier:

This function works in two cases: if the caught line contains output like ‘example_header time=12.32 mass=0.94 ..’ or if the line contains output like ‘example_header 12.32 0.94’ Please don’t the two cases.

You can give a ‘selected_header’ to catch any line that starts with that. Then the values will be put into a dictionary.

Tasks:
  • TODO: Think about exporting to numpy array or pandas instead of a defaultdict

  • TODO: rethink whether this function is necessary at all

  • TODO: check this function again

Parameters
  • output (str) – binary_c output string

  • selected_header (str) – string header of the output (the start of the line that you want to process)

Return type

dict

Returns

dictionary containing parameters as keys and lists for the values

binarycpython.utils.functions.extract_ensemble_json_from_string(binary_c_output)[source]

Function to extract the ensemble_json information from a raw binary_c output string

Parameters

binary_c_output (str) – raw binary_c output string

Return type

dict

Returns

JSON dictionary with the parsed ENSEMBLE_JSON data

binarycpython.utils.functions.filter_arg_dict(arg_dict)[source]

Function to filter out keys that contain values included in [‘NULL’, ‘Function’, ‘’]

This function is called by get_defaults()

Parameters

arg_dict (dict) – dictionary containing the argument + default key pairs of binary_c

Return type

dict

Returns

filtered dictionary (pairs with NULL and Function values are removed)

binarycpython.utils.functions.format_ensemble_results(ensemble_dictionary)[source]

Function to handle all the steps of formatting the ensemble output again.

Input:

ensemble_dictionary: dictionary containing all the ensemble results

binarycpython.utils.functions.format_number(number)[source]
binarycpython.utils.functions.get_arg_keys()[source]

Function that return the list of possible keys to give in the arg string. This function calls get_defaults()

Return type

list

Returns

list of all the parameters that binary_c accepts (and has default values for, since we call get_defaults())

binarycpython.utils.functions.get_defaults(filter_values=False)[source]

Function that calls the binaryc get args function and cast it into a dictionary.

All the values are strings

Parameters

filter_values (bool) – whether to filter out NULL and Function defaults.

Return type

dict

Returns

dictionary containing the parameter name as key and the parameter default as value

binarycpython.utils.functions.get_help(param_name='', print_help=True, fail_silently=False)[source]

Function that returns the help info for a given parameter, by interfacing with binary_c

Will check whether it is a valid parameter.

Binary_c will output things in the following order; - Did you mean? - binary_c help for variable - default - available macros

This function reads out that structure and catches the different components of this output

Tasks:
  • TODO: consider not returning None, but return empty dict

Parameters
  • param_name (str) – name of the parameter that you want info from. Will get checked whether its a valid parameter name

  • print_help (bool) – (optional, default = True) whether to print out the help information

  • fail_silently (bool) – (optional, default = False) Whether to print the errors raised if the

  • valid (parameter isn't) –

Return type

Optional[dict]

Returns

Dictionary containing the help info. This dictionary contains ‘parameter_name’, ‘parameter_value_input_type’, ‘description’, optionally ‘macros’

binarycpython.utils.functions.get_help_all(print_help=True)[source]

Function that reads out the output of the return_help_all API call to binary_c. This return_help_all binary_c returns all the information for the parameters, their descriptions and other properties. The output is categorised in sections.

Parameters

print_help (bool) – (optional, default = True) prints all the parameters and their descriptions.

Return type

dict

Returns

returns a dictionary containing dictionaries per section. These dictionaries contain the parameters and descriptions etc for all the parameters in that section

binarycpython.utils.functions.get_help_super(print_help=False, fail_silently=True)[source]

Function that first runs get_help_all, and then per argument also run the help function to get as much information as possible.

Parameters
  • print_help (bool) – (optional, default = False) Whether to print the information

  • fail_silently (bool) – (optional, default = True) Whether to fail silently or to print the errors

Return type

dict

Returns

dictionary containing all dictionaries per section, which then contain as much info as possible per parameter.

binarycpython.utils.functions.get_moe_di_stefano_dataset(options, verbosity=0)[source]

Function to get the default moe and di Stefano dataset or accept a user input.

binarycpython.utils.functions.get_size(obj, seen=None)[source]

Recursively finds size of objects

From https://github.com/bosswissam/pysize

binarycpython.utils.functions.get_username()[source]
binarycpython.utils.functions.handle_ensemble_string_to_json(raw_output)[source]

Function that deals with the raw output of the ensemble and creates a working JSON dictionary out of it.

Having this wrapper makes it easy to

Parameters

raw_output – raw output of the ensemble dump by binary_c

Returns

json.loads(raw_output, cls=binarycDecoder)

binarycpython.utils.functions.imports()[source]
binarycpython.utils.functions.inspect_dict(input_dict, indent=0, print_structure=True)[source]

Function to (recursively) inspect a (nested) dictionary. The object that is returned is a dictionary containing the key of the input_dict, but as value it will return the type of what the value would be in the input_dict

In this way we inspect the structure of these dictionaries, rather than the exact contents.

Parameters
  • input_dict (dict) – dictionary you want to inspect

  • print_structure (bool) – (optional, default = True)

  • indent (int) – (optional, default = 0) indent of the first output

Return type

dict

Returns

Dictionary that has the same structure as the input_dict, but as values it has the

type(input_dict[key]) (except if the value is a dict)

binarycpython.utils.functions.is_capsule(o)[source]

Function to tell whether object is a capsule

binarycpython.utils.functions.load_logfile(logfile)[source]

Experimental function that parses the generated log file of binary_c.

This function is not finished and shouldn’t be used yet.

Tasks:
  • TODO:

Parameters

logfile (-) – filename of the log file you want to parse

Returns:

Return type

None

binarycpython.utils.functions.make_build_text()[source]

Function to make build text

Return type

str

Returns

string containing information about the build and the git branch

binarycpython.utils.functions.mem_use()[source]

Return current process memory use in MB. (Takes no arguments) Note: this is per-thread only.

binarycpython.utils.functions.merge_dicts(dict_1, dict_2)[source]

Function to merge two dictionaries in a custom way.

Behaviour:

When dict keys are only present in one of either:
  • we just add the content to the new dict

When dict keys are present in both, we decide based on the value types how to combine them:
  • dictionaries will be merged by calling recursively calling this function again

  • numbers will be added

  • (opt) lists will be appended

  • In the case that the instances do not match: for now I will raise an error

Parameters
  • dict_1 (dict) – first dictionary

  • dict_2 (dict) – second dictionary

Return type

dict

Returns

Merged dictionary

binarycpython.utils.functions.multiply_values_dict(input_dict, factor)[source]

Function that goes over dictionary recursively and multiplies the value if possible by a factor

If the key equals “general_info”, the multiplication gets skipped

binarycpython.utils.functions.output_lines(output)[source]

Function that outputs the lines that were received from the binary_c run, but now as an iterator.

Parameters

output (str) – raw binary_c output

Return type

list

Returns

Iterator over the lines of the binary_c output

binarycpython.utils.functions.pad_output_distribution(dist, binwidth)[source]

Given a distribution, dist (a dictionary), which should be binned every binwidth (float), fill the distribution with zeros when there is no data. Note: this changes the data in place.

binarycpython.utils.functions.parse_binary_c_version_info(version_info_string)[source]

Function that parses the binary_c version info. Long function with a lot of branches

TODO: fix this function. stuff is missing: isotopes, macros, nucleosynthesis_sources

Parameters

version_info_string (str) – raw output of version_info call to binary_c

Returns

‘isotopes’ for isotope info, ‘argpairs’ for argument pair info (TODO: explain), ‘ensembles’ for ensemble settings/info, ‘macros’ for macros, ‘elements’ for atomic element info, ‘DTlimit’ for (TODO: explain), ‘nucleosynthesis_sources’ for nucleosynthesis sources, and ‘miscellaneous’ for all those that were not caught by the previous groups. ‘git_branch’, ‘git_build’, ‘revision’ and ‘email’ are also keys, but its clear what those contain.

Return type

Parsed version of the version info, which is a dictionary containing the keys

binarycpython.utils.functions.recursive_change_key_to_float(input_dict)[source]

Function to recursively change the key to float

This only works if the dict contains just sub-dicts or numbers/strings. Does not work with lists as values

binarycpython.utils.functions.recursive_change_key_to_string(input_dict)[source]

Function to recursively change the key back to a string but this time in a format that we decide

binarycpython.utils.functions.remove_file(file, verbosity=0)[source]

Function to remove files but with verbosity

Parameters
  • file (str) – full file path to the file that will be removed.

  • verbosity (int) – current verbosity level (Optional)

Return type

None

Returns

the path of a sub directory called binary_c_python in the TMP of the file system

binarycpython.utils.functions.return_binary_c_version_info(parsed=True)[source]

Function that returns the version information of binary_c. This function calls the function _binary_c_bindings.return_version_info()

Parameters

parsed (bool) – Boolean flag whether to parse the version_info output of binary_c. default = False

Return type

Union[str, dict]

Returns

Either the raw string of binary_c or a parsed version of this in the form of a nested dictionary

binarycpython.utils.functions.subtract_dicts(dict_1, dict_2)[source]

Function to subtract two dictionaries.

Only allows values to be either a dict or a numerical type

For the overlapping keys (key name present in both dicts):
When the keys are of the same type:
  • If the types are of numerical type: subtract the value at dict 2 from dict 1.

  • If the types are both dictionaries: call this function with the subdicts

When the keys are not of the same type:
  • if the keys are all of numerical types

For the unique keys:
  • if the key is from dict 1: adds the value to the new dict (be it numerical value or dict)

  • If the key is from dict 2: Adds the negative of its value in case of numerical type.

    if the type is a dict, the result of subtract_dicts({}, dict_2[key]) will be set

If the result is 0, the key will be removed from the resulting dict. If that results in an empty dict, the dict will be removed too.

Parameters
  • dict_1 (dict) – first dictionary

  • dict_2 (dict) – second dictionary

Return type

dict

Returns

Subtracted dictionary

binarycpython.utils.functions.temp_dir(*args)[source]

Function to create directory within the TMP directory of the file system

Makes use of os.makedirs exist_ok which requires python 3.2+

Parameters

arguments (function) – str input where each next input will be a child of the previous full_path. e.g. temp_dir(‘tests’, ‘grid’) will become ‘/tmp/binary_c_python/tests/grid’

Return type

str

Returns

the path of a sub directory called binary_c_python in the TMP of the file system

binarycpython.utils.functions.trem(dt, count, dn, n)[source]

Estimate time remaining (seconds) given a differential time and count (i.e. progress = $count/$n). $dt is the time since the last call, $count is the current progress count, $dn is the number run since the last call, and $n is the total number required.

binarycpython.utils.functions.update_dicts(dict_1, dict_2)[source]

Function to update dict_1 with values of dict_2 in a recursive way.

Behaviour:

When dict keys are only present in one of either:
  • we just add the content to the new dict

When dict keys are present in both, we decide based on the value types how to combine them:
  • value of dict2 will be taken

Parameters
  • dict_1 (dict) – first dictionary

  • dict_2 (dict) – second dictionary

Return type

dict

Returns

New dictionary with Updated values

binarycpython.utils.functions.verbose_print(message, verbosity, minimal_verbosity)[source]

Function that decides whether to print a message based on the current verbosity and its minimum verbosity

if verbosity is equal or higher than the minimum, then we print

Parameters
  • message (str) – message to print

  • verbosity (int) – current verbosity level

  • minimal_verbosity (int) – threshold verbosity above which to print

Return type

None

binarycpython.utils.functions.write_binary_c_parameter_descriptions_to_rst_file(output_file)[source]

Function that calls the get_help_super() to get the help text/descriptions for all the parameters available in that build. Writes the results to a .rst file that can be included in the docs.

Tasks:
  • TODO: add the specific version git branch, git build, git commit, and binary_c version to

    this document

Parameters

output_file (str) – name of the output .rst file containing the ReStructuredText formatted output of all the binary_c parameters.

Return type

None