functions module
Module containing most of the utility functions for the binarycpython package
Functions here are mostly functions used in other classes/functions, or useful functions for the user
- Tasks:
TODO: change all prints to verbose_prints
- class binarycpython.utils.functions.AutoVivificationDict[source]
Bases:
dict
Implementation of perl’s autovivification feature, by overriding the get item and the __iadd__ operator (https://docs.python.org/3/reference/datamodel.html?highlight=iadd#object.__iadd__)
This allows to set values within a subdict that might not exist yet:
Example
newdict = {} newdict[‘example’][‘mass’] += 10 print(newdict) >>> {‘example’: {‘mass’: 10}}
- class binarycpython.utils.functions.BinaryCEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]
Bases:
json.encoder.JSONEncoder
Encoding class function to attempt to convert things to strings.
- class binarycpython.utils.functions.Capturing(iterable=(), /)[source]
Bases:
list
Context manager to capture output and store it
- binarycpython.utils.functions.bin_data(value, binwidth)[source]
Function that bins the data
Uses the absolute value of binwidth
- class binarycpython.utils.functions.binarycDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)[source]
Bases:
json.decoder.JSONDecoder
Custom decoder to transform the numbers that are strings to actual floats
- binarycpython.utils.functions.binaryc_json_serializer(obj)[source]
Custom serialiser for binary_c to use when functions are present in the dictionary that we want to export.
Function objects will be turned into str representations of themselves
- Parameters
obj (
Any
) – The object that might not be serialisable- Return type
Any
- Returns
Either string representation of object if the object is a function, or the object itself
- binarycpython.utils.functions.call_binary_c_config(argument)[source]
Function to interface with the binary_c config file
- input:
argument: argument for the binary_c config
- Returns
raw output of binary_c-config
- class binarycpython.utils.functions.catchtime[source]
Bases:
object
Context manager to calculate time spent
- binarycpython.utils.functions.check_if_in_shell()[source]
Function to check whether the script is running from a shell
- binarycpython.utils.functions.conv_time_units(t)[source]
Converts time (t, in seconds, passing in as the only argument) to seconds, minutes or hours depending on its magnitude. Returns a tuple (t,units).
- binarycpython.utils.functions.convert_bytes(size)[source]
Function to return the size + a magnitude string
- binarycpython.utils.functions.convfloat(x)[source]
Convert scalar x to a float if we can, in which case return the float, otherwise just return x without changing it. Usually, x is a string, but could be anything that float() can handle without failure.
- binarycpython.utils.functions.count_keys_recursive(input_dict)[source]
Function to count the total number of keys in a dictionary
- binarycpython.utils.functions.create_arg_string(arg_dict, sort=False, filter_values=False)[source]
Function that creates the arg string for binary_c. Takes a dictionary containing the arguments and writes them to a string This string is missing the ‘binary_c ‘ at the start.
- Parameters
arg_dict (
dict
) – dictionarysort (
bool
) – (optional, default = False) Boolean whether to sort the order of the keys.filter_values (
bool
) – (optional, default = False) filters the input dict on keys that have NULL or function as value.
- Return type
str
- Returns
The string built up by combining all the key + value’s.
- binarycpython.utils.functions.create_hdf5(data_dir, name)[source]
- Function to create an hdf5 file from the contents of a directory:
settings file is selected by checking on files ending on settings
data files are selected by checking on files ending with .dat
TODO: fix missing settings files
- Parameters
data_dir (
str
) – directory containing the data files and settings filename (
str
) – name of hdf5file.
- Return type
None
- binarycpython.utils.functions.custom_sort_dict(input_dict)[source]
Returns a dictionary that is ordered, but can handle numbers better than normal OrderedDict
When the keys of the current dictionary are of mixed type, we first find all the unique types. Sort that list of type names. Then find the values that fit that type. Sort those and append them to the sorted keys list. This is done until all the keys are sorted.
All objects other than dictionary types are directly return as they are
- binarycpython.utils.functions.datalinedict(line, parameters)[source]
Convert a line of data to a more convenient dictionary. :param line = a line of data as a string: :param parameters = a list of the parameter names:
Note: if the parameter is a floating point number, it will be converted to Python’s float type.
- binarycpython.utils.functions.example_parse_output(output, selected_header)[source]
Function that parses output of binary_c. This version serves as an example and is quite detailed. Custom functions can be easier:
This function works in two cases: if the caught line contains output like ‘example_header time=12.32 mass=0.94 ..’ or if the line contains output like ‘example_header 12.32 0.94’ Please don’t the two cases.
You can give a ‘selected_header’ to catch any line that starts with that. Then the values will be put into a dictionary.
- Tasks:
TODO: Think about exporting to numpy array or pandas instead of a defaultdict
TODO: rethink whether this function is necessary at all
TODO: check this function again
- Parameters
output (
str
) – binary_c output stringselected_header (
str
) – string header of the output (the start of the line that you want to process)
- Return type
dict
- Returns
dictionary containing parameters as keys and lists for the values
- binarycpython.utils.functions.extract_ensemble_json_from_string(binary_c_output)[source]
Function to extract the ensemble_json information from a raw binary_c output string
- Parameters
binary_c_output (
str
) – raw binary_c output string- Return type
dict
- Returns
JSON dictionary with the parsed ENSEMBLE_JSON data
- binarycpython.utils.functions.filter_arg_dict(arg_dict)[source]
Function to filter out keys that contain values included in [‘NULL’, ‘Function’, ‘’]
This function is called by get_defaults()
- Parameters
arg_dict (
dict
) – dictionary containing the argument + default key pairs of binary_c- Return type
dict
- Returns
filtered dictionary (pairs with NULL and Function values are removed)
- binarycpython.utils.functions.format_ensemble_results(ensemble_dictionary)[source]
Function to handle all the steps of formatting the ensemble output again.
- Input:
ensemble_dictionary: dictionary containing all the ensemble results
- binarycpython.utils.functions.get_arg_keys()[source]
Function that return the list of possible keys to give in the arg string. This function calls get_defaults()
- Return type
list
- Returns
list of all the parameters that binary_c accepts (and has default values for, since we call get_defaults())
- binarycpython.utils.functions.get_defaults(filter_values=False)[source]
Function that calls the binaryc get args function and cast it into a dictionary.
All the values are strings
- Parameters
filter_values (
bool
) – whether to filter out NULL and Function defaults.- Return type
dict
- Returns
dictionary containing the parameter name as key and the parameter default as value
- binarycpython.utils.functions.get_help(param_name='', print_help=True, fail_silently=False)[source]
Function that returns the help info for a given parameter, by interfacing with binary_c
Will check whether it is a valid parameter.
Binary_c will output things in the following order; - Did you mean? - binary_c help for variable - default - available macros
This function reads out that structure and catches the different components of this output
- Tasks:
TODO: consider not returning None, but return empty dict
- Parameters
param_name (
str
) – name of the parameter that you want info from. Will get checked whether its a valid parameter nameprint_help (
bool
) – (optional, default = True) whether to print out the help informationfail_silently (
bool
) – (optional, default = False) Whether to print the errors raised if thevalid (parameter isn't) –
- Return type
Optional
[dict
]- Returns
Dictionary containing the help info. This dictionary contains ‘parameter_name’, ‘parameter_value_input_type’, ‘description’, optionally ‘macros’
- binarycpython.utils.functions.get_help_all(print_help=True)[source]
Function that reads out the output of the return_help_all API call to binary_c. This return_help_all binary_c returns all the information for the parameters, their descriptions and other properties. The output is categorised in sections.
- Parameters
print_help (
bool
) – (optional, default = True) prints all the parameters and their descriptions.- Return type
dict
- Returns
returns a dictionary containing dictionaries per section. These dictionaries contain the parameters and descriptions etc for all the parameters in that section
- binarycpython.utils.functions.get_help_super(print_help=False, fail_silently=True)[source]
Function that first runs get_help_all, and then per argument also run the help function to get as much information as possible.
- Parameters
print_help (
bool
) – (optional, default = False) Whether to print the informationfail_silently (
bool
) – (optional, default = True) Whether to fail silently or to print the errors
- Return type
dict
- Returns
dictionary containing all dictionaries per section, which then contain as much info as possible per parameter.
- binarycpython.utils.functions.get_moe_di_stefano_dataset(options, verbosity=0)[source]
Function to get the default moe and di Stefano dataset or accept a user input.
- binarycpython.utils.functions.handle_ensemble_string_to_json(raw_output)[source]
Function that deals with the raw output of the ensemble and creates a working JSON dictionary out of it.
Having this wrapper makes it easy to
- Parameters
raw_output – raw output of the ensemble dump by binary_c
- Returns
json.loads(raw_output, cls=binarycDecoder)
- binarycpython.utils.functions.inspect_dict(input_dict, indent=0, print_structure=True)[source]
Function to (recursively) inspect a (nested) dictionary. The object that is returned is a dictionary containing the key of the input_dict, but as value it will return the type of what the value would be in the input_dict
In this way we inspect the structure of these dictionaries, rather than the exact contents.
- Parameters
input_dict (
dict
) – dictionary you want to inspectprint_structure (
bool
) – (optional, default = True)indent (
int
) – (optional, default = 0) indent of the first output
- Return type
dict
- Returns
- Dictionary that has the same structure as the input_dict, but as values it has the
type(input_dict[key]) (except if the value is a dict)
- binarycpython.utils.functions.load_logfile(logfile)[source]
Experimental function that parses the generated log file of binary_c.
This function is not finished and shouldn’t be used yet.
- Tasks:
TODO:
- Parameters
logfile (-) – filename of the log file you want to parse
Returns:
- Return type
None
- binarycpython.utils.functions.make_build_text()[source]
Function to make build text
- Return type
str
- Returns
string containing information about the build and the git branch
- binarycpython.utils.functions.mem_use()[source]
Return current process memory use in MB. (Takes no arguments) Note: this is per-thread only.
- binarycpython.utils.functions.merge_dicts(dict_1, dict_2)[source]
Function to merge two dictionaries in a custom way.
Behaviour:
- When dict keys are only present in one of either:
we just add the content to the new dict
- When dict keys are present in both, we decide based on the value types how to combine them:
dictionaries will be merged by calling recursively calling this function again
numbers will be added
(opt) lists will be appended
In the case that the instances do not match: for now I will raise an error
- Parameters
dict_1 (
dict
) – first dictionarydict_2 (
dict
) – second dictionary
- Return type
dict
- Returns
Merged dictionary
- binarycpython.utils.functions.multiply_values_dict(input_dict, factor)[source]
Function that goes over dictionary recursively and multiplies the value if possible by a factor
If the key equals “general_info”, the multiplication gets skipped
- binarycpython.utils.functions.output_lines(output)[source]
Function that outputs the lines that were received from the binary_c run, but now as an iterator.
- Parameters
output (
str
) – raw binary_c output- Return type
list
- Returns
Iterator over the lines of the binary_c output
- binarycpython.utils.functions.pad_output_distribution(dist, binwidth)[source]
Given a distribution, dist (a dictionary), which should be binned every binwidth (float), fill the distribution with zeros when there is no data. Note: this changes the data in place.
- binarycpython.utils.functions.parse_binary_c_version_info(version_info_string)[source]
Function that parses the binary_c version info. Long function with a lot of branches
TODO: fix this function. stuff is missing: isotopes, macros, nucleosynthesis_sources
- Parameters
version_info_string (
str
) – raw output of version_info call to binary_c- Returns
‘isotopes’ for isotope info, ‘argpairs’ for argument pair info (TODO: explain), ‘ensembles’ for ensemble settings/info, ‘macros’ for macros, ‘elements’ for atomic element info, ‘DTlimit’ for (TODO: explain), ‘nucleosynthesis_sources’ for nucleosynthesis sources, and ‘miscellaneous’ for all those that were not caught by the previous groups. ‘git_branch’, ‘git_build’, ‘revision’ and ‘email’ are also keys, but its clear what those contain.
- Return type
Parsed version of the version info, which is a dictionary containing the keys
- binarycpython.utils.functions.recursive_change_key_to_float(input_dict)[source]
Function to recursively change the key to float
This only works if the dict contains just sub-dicts or numbers/strings. Does not work with lists as values
- binarycpython.utils.functions.recursive_change_key_to_string(input_dict)[source]
Function to recursively change the key back to a string but this time in a format that we decide
- binarycpython.utils.functions.remove_file(file, verbosity=0)[source]
Function to remove files but with verbosity
- Parameters
file (
str
) – full file path to the file that will be removed.verbosity (
int
) – current verbosity level (Optional)
- Return type
None
- Returns
the path of a sub directory called binary_c_python in the TMP of the file system
- binarycpython.utils.functions.return_binary_c_version_info(parsed=True)[source]
Function that returns the version information of binary_c. This function calls the function _binary_c_bindings.return_version_info()
- Parameters
parsed (
bool
) – Boolean flag whether to parse the version_info output of binary_c. default = False- Return type
Union
[str
,dict
]- Returns
Either the raw string of binary_c or a parsed version of this in the form of a nested dictionary
- binarycpython.utils.functions.subtract_dicts(dict_1, dict_2)[source]
Function to subtract two dictionaries.
Only allows values to be either a dict or a numerical type
- For the overlapping keys (key name present in both dicts):
- When the keys are of the same type:
If the types are of numerical type: subtract the value at dict 2 from dict 1.
If the types are both dictionaries: call this function with the subdicts
- When the keys are not of the same type:
if the keys are all of numerical types
- For the unique keys:
if the key is from dict 1: adds the value to the new dict (be it numerical value or dict)
- If the key is from dict 2: Adds the negative of its value in case of numerical type.
if the type is a dict, the result of subtract_dicts({}, dict_2[key]) will be set
If the result is 0, the key will be removed from the resulting dict. If that results in an empty dict, the dict will be removed too.
- Parameters
dict_1 (
dict
) – first dictionarydict_2 (
dict
) – second dictionary
- Return type
dict
- Returns
Subtracted dictionary
- binarycpython.utils.functions.temp_dir(*args)[source]
Function to create directory within the TMP directory of the file system
Makes use of os.makedirs exist_ok which requires python 3.2+
- Parameters
arguments (function) – str input where each next input will be a child of the previous full_path. e.g. temp_dir(‘tests’, ‘grid’) will become ‘/tmp/binary_c_python/tests/grid’
- Return type
str
- Returns
the path of a sub directory called binary_c_python in the TMP of the file system
- binarycpython.utils.functions.trem(dt, count, dn, n)[source]
Estimate time remaining (seconds) given a differential time and count (i.e. progress = $count/$n). $dt is the time since the last call, $count is the current progress count, $dn is the number run since the last call, and $n is the total number required.
- binarycpython.utils.functions.update_dicts(dict_1, dict_2)[source]
Function to update dict_1 with values of dict_2 in a recursive way.
Behaviour:
- When dict keys are only present in one of either:
we just add the content to the new dict
- When dict keys are present in both, we decide based on the value types how to combine them:
value of dict2 will be taken
- Parameters
dict_1 (
dict
) – first dictionarydict_2 (
dict
) – second dictionary
- Return type
dict
- Returns
New dictionary with Updated values
- binarycpython.utils.functions.verbose_print(message, verbosity, minimal_verbosity)[source]
Function that decides whether to print a message based on the current verbosity and its minimum verbosity
if verbosity is equal or higher than the minimum, then we print
- Parameters
message (
str
) – message to printverbosity (
int
) – current verbosity levelminimal_verbosity (
int
) – threshold verbosity above which to print
- Return type
None
- binarycpython.utils.functions.write_binary_c_parameter_descriptions_to_rst_file(output_file)[source]
Function that calls the get_help_super() to get the help text/descriptions for all the parameters available in that build. Writes the results to a .rst file that can be included in the docs.
- Tasks:
- TODO: add the specific version git branch, git build, git commit, and binary_c version to
this document
- Parameters
output_file (
str
) – name of the output .rst file containing the ReStructuredText formatted output of all the binary_c parameters.- Return type
None