The utilities module¶
This module contains convenience functions for quick setup of common tasks.
Table of Contents
Get a logger¶
The get_logger()
function is a convenience function to setup logging output. It
will return a named logger, which can be used inside programs. The function has the
ability to setup logging both to a terminal, to a log file, including setting up log
rotation and for sending out email on log message at warning level or above.
get_logger
usage examples¶
Logging to terminal and send emails on warnings and above (default)¶
To get a named logger that will output logging information to the terminal and send emails on warnings and above, do the following:
from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('name_of_my_logger')
where the name_of_my_logger
should be some descriptive name for
what the program/script does e.g. “coffee_machine_count_monitor”.
From the returned LOGGER
, information can now be logged via the
usual .info()
, .warning()
methods etc.
To turn logging to the terminal of (and only use the other configured
logging handlers), set the optional boolean terminal_log
parameter
to False
.
The email notification on warnings and above is on-by-default and is
controlled by the two optional boolean parameters
email_on_warnings
and email_on_errors
. It will send emails on
logged warnings to the warnings list and on logged errors (and above)
to the error list.
Rotating file logger¶
To get a named logger that also logs to a file do:
from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('name_of_my_logger', file_log=True)
The log will be written to the file name_of_my_logger.log
. The
file name can be changed via the option file_name
, the same way as
the maximum log file size and number of saved backups can be changed,
as documented below
.
Getting the logger to send emails on un-caught exceptions¶
To also make the logger send en email, containing any un-caught exception, do the following:
from PyExpLabSys.common.utilities import get_logger
LOGGER = get_logger('Test logger')
def main():
pass # All your main code that might raise exceptions
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
pass # Shut down code here
except Exception:
LOGGER.exception("Main program failed")
# Possibly shut down code here
raise
Note
The argument to LOGGER.exception
is a string, just like all other LOGGER
methods. All the exception information, like the traceback, line number etc., is
picked up automatically by the logger. Also note, that the raise
will re-raise the
caught exception, to make sure that the program fails like it is supposed to. That it
is the caught exception that is re-raised, is implicit when using raise without
arguments in an except clause.
Note
In the example there is a seperate except
for the
KeyboardInterrupt
exception, to make it possible to use
keyboard interrupts to shut the program down, without sending an
exception email about it.
Auto-generated module documentation¶
This module contains a convenience function for easily setting up a
logger with the logging
module.
This module uses the following settings from the Settings
class:
- util_log_warning_email
- util_log_error_email
- util_log_mail_host
- util_log_max_emails_per_period (defaults to 5)
- util_log_email_throttle_time (defaults to 86400s = 1day)
- util_log_backlog_limit (defaults to 250)
Note
All of these settings are at present read from the settings module at import time, so if it is desired to modify them at run time, it should be done before import
-
PyExpLabSys.common.utilities.
SETTINGS
= <PyExpLabSys.settings.Settings object>¶ The
Settings
object used in this module
-
PyExpLabSys.common.utilities.
WARNING_EMAIL
= 'fake@non.com'¶ The email list warning emails are sent to
-
PyExpLabSys.common.utilities.
ERROR_EMAIL
= 'fake@non.com'¶ The email list error emails are sent to
-
PyExpLabSys.common.utilities.
MAIL_HOST
= 'none'¶ The email host used to send emails on logged warnings and errors
-
PyExpLabSys.common.utilities.
MAX_EMAILS_PER_PERIOD
= 5¶ The maximum number of emails the logger will send in
EMAIL_THROTTLE_TIME
-
PyExpLabSys.common.utilities.
EMAIL_THROTTLE_TIME
= 86400¶ The time period that the numbers of emails will be limited within
-
PyExpLabSys.common.utilities.
EMAIL_BACKLOG_LIMIT
= 250¶ The maximum number of messages in the email backlog that will be sent when the next email is let through
-
PyExpLabSys.common.utilities.
get_logger
(name, level='INFO', terminal_log=True, file_log=False, file_name=None, file_max_bytes=1048576, file_backup_count=1, email_on_warnings=True, email_on_errors=True)[source]¶ Setup and return a program logger
This is meant as a logger to be used in a top level program/script. The logger is set up for with terminal, file and email handlers if requested.
Parameters: - name (str) – The name of the logger, e.g: ‘my_fancy_program’. Passing in an empty string will return the root logger. See note below.
- level (str) – The level for the logger. Can be either
'DEBUG'
,'INFO'
,'WARNING'
,'ERROR'
or'CRITICAL'
. Seelogging
for details. Default is'INFO'
. - terminal_log (bool) – If
True
then logging to a terminal will be activated. Default isTrue
. - file_log (bool) – If
True
then logging to a file, with log rotation, will be activated. Iffile_name
is not given, thenname + '.log'
will be used. Default isFalse
. - file_name (str) – Optional file name to log to
- file_max_size (int) – The maximum size of the log file in bytes. The
default is
1048576
(1MB), which corresponds to roughly 10000 lines of log per file. - file_backup_count (int) – The number of backup logs to keep. The default
is
1
. - email_on_warnings (bool) – Whether to send an email to the
WARNING_EMAIL
email list if a warning is logged. The default isTrue
. - email_on_error (bool) – Whether to send en email to the
ERROR_EMAIL
email list if an error (or any logging level above) is logged. The default isTrue
.
Returns: A logger module with the requested setup
Return type: Note
Passing in the empty string as the
name
, will return the root logger. That means that all other library loggers will inherit the level and handlers from this logger, which may potentially result in a lot of output. Seeactivate_library_logging()
for a way to activate the library loggers from PyExpLabSys in a more controlled manner.
-
PyExpLabSys.common.utilities.
get_library_logger_names
()[source]¶ Return all loggers currently configured in PyExpLabSys
-
PyExpLabSys.common.utilities.
print_library_logger_names
()[source]¶ Nicely printout all loggers currently configured in PyExpLabSys
-
PyExpLabSys.common.utilities.
activate_library_logging
(logger_name, logger_to_inherit_from=None, level=None, terminal_log=True, file_log=False, file_name=None, file_max_bytes=1048576, file_backup_count=1, email_on_warnings=True, email_on_errors=True)[source]¶ Activate logging for a PyExpLabSys library logger
Parameters: - logger_name (str) – The name of the logger to activate, as returned by
get_library_logger_names()
- logger_to_inherit_from (logging.Logger) – (Optional) If this is set, the library
logger will simply share the handlers that are present in this logger. The
library to be activated will also inherit the level from this logger, unless
level
is set, in which case it will override. In case neitherlevel
nor the level onlogger_to_inherit_from
is set, the level will not be changed. - level (str) – (Optional) See docstring for
get_logger()
. Iflogger_to_inherit_from
is not set, it will default to ‘info’. - terminal_log (bool) – See docstring for
get_logger()
- file_log (bool) – See docstring for
get_logger()
- file_name (str) – See docstring for
get_logger()
- file_max_size (int) – See docstring for
get_logger()
- file_backup_count (int) – See docstring for
get_logger()
- email_on_warnings (bool) – See docstring for
get_logger()
- email_on_error (bool) – See docstring for
get_logger()
- logger_name (str) – The name of the logger to activate, as returned by
-
class
PyExpLabSys.common.utilities.
CustomSMTPHandler
(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None, timeout=5.0)[source]¶ Bases:
logging.handlers.SMTPHandler
PyExpLabSys modified SMTP handler
-
class
PyExpLabSys.common.utilities.
CustomSMTPWarningHandler
(mailhost, fromaddr, toaddrs, subject, credentials=None, secure=None, timeout=5.0)[source]¶ Bases:
PyExpLabSys.common.utilities.CustomSMTPHandler
Custom SMTP handler to emit record only if: warning =< level < error