The loggers module

The logger module contains classes that hide some of all the repeated code associated with sending data to the data base. The main component is the ContinousLogger class, which is used to send continuously logged data to the data base.

The continuous logger

The ContinuousLogger class is meant to do as much of the manual work related to logging a parameter continuously to the database as possible. The main features are:

  • Simple single method usage. After the class is instantiated, a single call to enqueue_point() or enqueue_point_now() is all that is needed to log a point. No manual database query manipulation is required.
  • Resistent to network or database down time. The class implements a queue for the data, from which points will only be removed if they are successfully handed of to the data base and while it is not possible to hand the data of, it will be stored in memory.

Warning

The resilience against network downtime has only been tested for the way it will fail if you disable the network from the machine it is running on. Different kinds of failures may produce different kinds of failure modes. If you encounter a failure mode that the class did not recover from you should report it as a bug.

Todo

Write that it uses new style database layout and refer to that section.

Usage Example

If code already exists to retrieve the data (e.g. a driver to interface a piece of equipment with), writing a data logger can be reduced to as little as the following:

from PyExpLabSys.common.loggers import ContinuousLogger

db_logger = ContinuousLogger(table='dateplots_dummy',
                             username='dummy', password='dummy',
                             measurement_codenames = ['dummy_sine_one'])
db_logger.start()

# Initialize variable for the logging condition
while True:
    new_value = driver.get_value()
    if contition_to_log_is_true:
        db_logger.enqueue_point_now('dummy_sine_one', new_value)

or if it is preferred to keep track of the timestamp manually:

import time
from PyExpLabSys.common.loggers import ContinuousLogger

# Initiate the logger to write to the dateplots_dummy table, with usernam
db_logger = ContinuousLogger(table='dateplots_dummy',
                             username='dummy', password='dummy',
                             measurement_codenames = ['dummy_sine_one'])
db_logger.start()

# Initialize variable for the logging condition
while True:
    new_value = driver.get_value()
    now = time.time()
    if contition_to_log_is_true:
        db_logger.enqueue_point('dummy_sine_one', (now, new_value))

Auto-generated module documentation

This module contains convinience classes for database logging. The classes implement queues to contain the data before of loading to the database to ensure against network or server problems.

class PyExpLabSys.common.loggers.NoneResponse[source]

Bases: object

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

PyExpLabSys.common.loggers.NONE_RESPONSE = <PyExpLabSys.common.loggers.NoneResponse object>

Module variable used to indicate a none response, currently is an instance if NoneResponse

class PyExpLabSys.common.loggers.InterruptableThread(cursor, query)[source]

Bases: threading.Thread

Class to run a MySQL query with a time out

__init__(cursor, query)[source]

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is the argument tuple for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

run()[source]

Start the thread

PyExpLabSys.common.loggers.timeout_query(cursor, query, timeout_duration=3)[source]

Run a mysql query with a timeout

Parameters:
  • cursor (MySQL cursor) – The database cursor
  • query (str) – The query to execute
  • timeout_duration (int) – The timeout duration
Returns
(tuple or NONE_RESPONSE): A tuple of results from the query or
NONE_RESPONSE if the query timed out
exception PyExpLabSys.common.loggers.StartupException(*args, **kwargs)[source]

Bases: Exception

Exception raised when the continous logger fails to start up

__init__(*args, **kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

class PyExpLabSys.common.loggers.ContinuousLogger(table, username, password, measurement_codenames, dequeue_timeout=1, reconnect_waittime=60, dsn=None)[source]

Bases: threading.Thread

A logger for continous data as a function of datetime. The class can ONLY be used with the new layout of tables for continous data, where there is only one table per setup, as apposed to the old layout where there was one table per measurement type per setup. The class sends data to the cinfdata database at host servcinf-sql.

Variables:
  • host – Database host, value is servcinf-sql.
  • database – Database name, value is cinfdata.
__init__(table, username, password, measurement_codenames, dequeue_timeout=1, reconnect_waittime=60, dsn=None)[source]

Initialize the continous logger

Parameters:
  • table (str) – The table to log data to
  • username (str) – The MySQL username (must have write rights to table)
  • password (str) – The password for user in the database
  • measurement_codenames (list) – List of measurement codenames that this logger will send data to
  • dequeue_timeout (float) – The timeout (in seconds) for dequeueing an element, which also constitutes the max time to shutdown after the thread has been asked to stop. Default is 1.
  • reconnect_waittime (float) – Time to wait (in seconds) in between attempts to re-connect to the MySQL database, if the connection has been lost
  • dsn (str) – DSN name of ODBC connection, used on Windows only
Raises:

StartupException – If it is not possible to start the database connection or translate the code names

stop()[source]

Stop the thread

run()[source]

Start the thread. Must be run before points are added.

enqueue_point_now(codename, value)[source]

Add a point to the queue and use the current time as the time

Parameters:
  • codename (str) – The measurement codename that this point will be saved under
  • value (float) – The value to be logged
Returns:

The Unixtime used

Return type:

float

enqueue_point(codename, point)[source]

Add a point to the queue

Parameters:
  • codename (str) – The measurement codename that this point will be saved under
  • point (iterable) – Current point as a list (or tuple) of 2 floats: [x, y]