The utilities module

This module contains convenience functions for quick setup of common tasks.

Get a logger

The get_logger() function is a convenience function to setup logging output. It will return a named logger, which can be used inside programs. The function has the ability to setup logging both to a terminal, to a log file, including setting up log rotation and for sending out email on log message at warning level or above.

get_logger usage examples

Logging to terminal and send emails on warnings and above (default)

To get a named logger that will output logging information to the terminal and send emails on warnings and above, do the following:

from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('name_of_my_logger')

where the name_of_my_logger should be some descriptive name for what the program/script does e.g. “coffee_machine_count_monitor”.

From the returned LOGGER, information can now be logged via the usual .info(), .warning() methods etc.

To turn logging to the terminal of (and only use the other configured logging handlers), set the optional boolean terminal_log parameter to False.

The email notification on warnings and above is on-by-default and is controlled by the two optional boolean parameters email_on_warnings and email_on_errors. It will send emails on logged warnings to the warnings list and on logged errors (and above) to the error list.

Rotating file logger

To get a named logger that also logs to a file do:

from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('name_of_my_logger', file_log=True)

The log will be written to the file name_of_my_logger.log. The file name can be changed via the option file_name, the same way as the maximum log file size and number of saved backups can be changed, as documented below.

Getting the logger to send emails on un-caught exceptions

To also make the logger send en email, containing any un-caught exception, do the following:

from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('Test logger')

def main():
    pass  # All your main code that might raise exceptions

if __name__ == '__main__':
    try:
        main()
    except KeyboardInterrupt:
        pass  # Shut down code here
    except Exception:
        LOGGER.exception("Main program failed")
        # Possibly shut down code here
        raise

Note

The argument to LOGGER.exception is a string, just like all other LOGGER methods. All the exception information, like the traceback, line number etc., is picked up automatically by the logger. Also note, that the raise will re-raise the caught exception, to make sure that the program fails like it is supposed to. That it is the caught exception that is re-raised, is implicit when using raise without arguments in an except clause.

Note

In the example there is a seperate except for the KeyboardInterrupt exception, to make it possible to use keyboard interrupts to shut the program down, without sending an exception email about it.

Auto-generated module documentation

This module contains a convenience function for easily setting up a logger with the logging module.

This module uses the following settings from the Settings class:

  • util_log_warning_email
  • util_log_error_email
  • util_log_mail_host
  • util_log_max_emails_per_period (defaults to 5)
  • util_log_email_throttle_time (defaults to 86400s = 1day)
  • util_log_backlog_limit (defaults to 250)

Note

All of these settings are at present read from the settings module at import time, so if it is desired to modify them at run time, it should be done before import

PyExpLabSys.common.utilities.SETTINGS = <PyExpLabSys.settings.Settings object>

The Settings object used in this module

PyExpLabSys.common.utilities.WARNING_EMAIL = 'fake@non.com'

The email list warning emails are sent to

PyExpLabSys.common.utilities.ERROR_EMAIL = 'fake@non.com'

The email list error emails are sent to

PyExpLabSys.common.utilities.MAIL_HOST = 'none'

The email host used to send emails on logged warnings and errors

PyExpLabSys.common.utilities.MAX_EMAILS_PER_PERIOD = 5

The maximum number of emails the logger will send in EMAIL_THROTTLE_TIME

PyExpLabSys.common.utilities.EMAIL_THROTTLE_TIME = 86400

The time period that the numbers of emails will be limited within

PyExpLabSys.common.utilities.EMAIL_BACKLOG_LIMIT = 250

The maximum number of messages in the email backlog that will be sent when the next email is let through

PyExpLabSys.common.utilities.get_logger(name, level='INFO', terminal_log=True, file_log=False, file_name=None, file_max_bytes=1048576, file_backup_count=1, email_on_warnings=True, email_on_errors=True)[source]

Setup and return a program logger

This is meant as a logger to be used in a top level program/script. The logger is set up for with terminal, file and email handlers if requested.

Parameters:
  • name (str) – The name of the logger, e.g: ‘my_fancy_program’. Passing in an empty string will return the root logger. See note below.
  • level (str) – The level for the logger. Can be either 'DEBUG', 'INFO', 'WARNING', 'ERROR' or 'CRITICAL'. See logging for details. Default is 'INFO'.
  • terminal_log (bool) – If True then logging to a terminal will be activated. Default is True.
  • file_log (bool) – If True then logging to a file, with log rotation, will be activated. If file_name is not given, then name + '.log' will be used. Default is False.
  • file_name (str) – Optional file name to log to
  • file_max_size (int) – The maximum size of the log file in bytes. The default is 1048576 (1MB), which corresponds to roughly 10000 lines of log per file.
  • file_backup_count (int) – The number of backup logs to keep. The default is 1.
  • email_on_warnings (bool) – Whether to send an email to the WARNING_EMAIL email list if a warning is logged. The default is True.
  • email_on_error (bool) – Whether to send en email to the ERROR_EMAIL email list if an error (or any logging level above) is logged. The default is True.
Returns:

A logger module with the requested setup
Return type:

logging.Logger

Note

Passing in the empty string as the name, will return the root logger. That means that all other library loggers will inherit the level and handlers from this logger, which may potentially result in a lot of output. See activate_library_logging() for a way to activate the library loggers from PyExpLabSys in a more controlled manner.

PyExpLabSys.common.utilities.get_library_logger_names()[source]

Return all loggers currently configured in PyExpLabSys

PyExpLabSys.common.utilities.print_library_logger_names()[source]

Nicely printout all loggers currently configured in PyExpLabSys

PyExpLabSys.common.utilities.activate_library_logging(logger_name, logger_to_inherit_from=None, level=None, terminal_log=True, file_log=False, file_name=None, file_max_bytes=1048576, file_backup_count=1, email_on_warnings=True, email_on_errors=True)[source]

Activate logging for a PyExpLabSys library logger

Parameters:
  • logger_name (str) – The name of the logger to activate, as returned by get_library_logger_names()
  • logger_to_inherit_from (logging.Logger) – (Optional) If this is set, the library logger will simply share the handlers that are present in this logger. The library to be activated will also inherit the level from this logger, unless level is set, in which case it will override. In case neither level nor the level on logger_to_inherit_from is set, the level will not be changed.
  • level (str) – (Optional) See docstring for get_logger(). If logger_to_inherit_from is not set, it will default to ‘info’.
  • terminal_log (bool) – See docstring for get_logger()
  • file_log (bool) – See docstring for get_logger()
  • file_name (str) – See docstring for get_logger()
  • file_max_size (int) – See docstring for get_logger()
  • file_backup_count (int) – See docstring for get_logger()
  • email_on_warnings (bool) – See docstring for get_logger()
  • email_on_error (bool) – See docstring for get_logger()
class PyExpLabSys.common.utilities.CustomSMTPHandler(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None, timeout=5.0)[source]

Bases: logging.handlers.SMTPHandler

PyExpLabSys modified SMTP handler

emit(record)[source]

Custom emit that throttles the number of email sent

getSubject(record)[source]

Returns subject with hostname

class PyExpLabSys.common.utilities.CustomSMTPWarningHandler(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None, timeout=5.0)[source]

Bases: PyExpLabSys.common.utilities.CustomSMTPHandler

Custom SMTP handler to emit record only if: warning =< level < error

emit(record)[source]

Custom emit that checks if: warning =< level < error

PyExpLabSys.common.utilities.call_spec_string()[source]

Return the argument names and values of the method or function it was called from as a string

Returns:
The argument string, e.g:
(name=’hello’, codenames=[‘aa’, ‘bb’], port=8000)
Return type:str