The sockets module

The data sockets are convenience classes that make it easier to send data back and forth between machines. All the data sockets are socket servers, i.e. they handle requests, and to interact with them it is necessary to work as a client. The main purpose of these sockets is to hide the complexity and present a easy-to-use interface while performing e.g. error checking in the background.

The sockets are divided into push and pull sockets, which are intended to either pull data from or pushing data to.

The main advantages of the pull sockets are:

• Simple usage: After e.g. the DateDataPullSocket or DataPullSocket class is instantiated, a single call to the set_point method is all that is needed to make a point available via the socket.
• Codename based: After instantiation, the different data slots are referred to by codenames. This makes code easier to read and help to prevent e.g. indexing errors.
• Timeout safety to prevent serving obsolete data: The class can be instantiated with a timeout for each measurement type. If the available point is too old an error message will be served.

The main advantages of the push socket are:

• Simple usage: If all that is required is to receive data in a variable like manner, both the last and the updated variable values can be accessed via the DataPushSocket.last and DataPushSocket.updated properties.
• Flexible: The DataPushSocket offers a lot of functionality around what actions are performed when a data set is received, including enqueue it or calling a callback function with the data set as an argument.

Table of Contents

• The sockets module
  • Examples
    • DateDataPullSocket make data available (network variable)
    • DataPushSocket, send data (network variable)
    • DataPushSocket, see all data sets received (enqueue them)
    • DataPushSocket, make socket call function on reception (callback)
    • DataPushSocket, control class and send return values back (callback with return)
  • Port defaults
  • Inheritance
  • Status of a socket server
    • Socket server status
    • System status
  • Auto-generated module documentation

Examples

In the following examples it is assumed, that all other code that is needed, such as e.g. an equipment driver, already exists, and the places where such code is needed, is filled in with dummy code.

DateDataPullSocket make data available (network variable)

Making data available on the network for pulling can be achieved with:

from PyExpLabSys.common.sockets import DateDataPullSocket

# Create a data socket with timeouts and start the socket server
name = 'Last shot usage data from the giant laser on the moon'
codenames = ['moon_laser_power', 'moon_laser_duration']
moon_socket = DateDataPullSocket(name, codenames, timeouts=[1.0, 0.7])
moon_socket.start()

try:
    while True:
        power, duration = laser.get_values()
        # To set a variable use its codename
        moon_socket.set_point_now('moon_laser_power', power)
        moon_socket.set_point_now('moon_laser_duration', duration)
except KeyboardInterrupt:
    # Stop the socket server
    moon_socket.stop()

or if it is preferred to keep track of the timestamp manually:

try:
    while True:
        now = time.time()
        power, duration = driver.get_values()
        moon_socket.set_point('moon_laser_power', (now, power))
        moon_socket.set_point('moon_laser_duration', (now, duration))
except KeyboardInterrupt:
    # Stop the socket server
    moon_socket.stop()

A few things to note from the examples. The port number used for the socket is 9000, which is the default for this type is socket (see Port defaults). The two measurements have been setup to have different timeouts (maximum ages), which is in seconds by the way, but it could also be set up to be the same, and if it is the same, it can be supplied as just one float timeouts=0.7 instead of a list of floats. For the sockets, the codenames should be kept relatively short, but for data safety reasons, they should contain an unambiguous reference to the setup, i.e. the ‘moon_laser’ part.

Client side

The client can be set up in the following manner:

import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# hostname can something like 'rasppi250', if that happens to be the one
# located on the moon
host_port = ('moon_raspberry_pi', 9000)

Command and data examples

With the initialization of the client as above, it is now possible to send the socket different commands and get appropriate responses. In the following, the different commands are listed, along with how to send it, receive (and decode) the reply.

The name command

Used to get the name of the socket server, which can be used to make sure that data is being pulled from the correct location:

command = 'name'
sock.sendto(command, host_port)
data = sock.recv(2048)

at which point the data variable, which contains the reply, will contain the string 'Last shot usage data from the giant laser on the moon'.

The json_wn command

Tip

The ‘wn’ suffix is short for ‘with names’ and is used for all the sockets to indicate that data is sent or received prefixed with names of the particular data channel

This command is used to get all the latest data encoded as json. It will retrieve all the data as a dictionary where the keys are the names, encoded as json for transport:

import json
command = 'json_wn'
sock.sendto(command, host_port)
data = json.loads(sock.recv(2048))

at which point the data variable will contain a dict like this one:

{u'moon_laser_power': [1414150015.697648, 47.0], u'moon_laser_duration': [1414150015.697672, 42.0]}

The codenames_json and json commands

It is also possible to decouple the codenames. A possible use case might be to produce a plot of the data. In such a case, the names are really only needed when setting up the plot, and then afterwards the data should just arrive in the same order, to add points to the graph. This is exactly what these two command do:

import json

# Sent only once
command = 'codenames_json'
sock.sendto(command, host_port)
codenames = json.loads(sock.recv(2048))

# Sent repeatedly
command = 'json'
sock.sendto(command, host_port)
data = json.loads(sock.recv(2048))

after which the codenames variable would contain a list of codenames:

[u'moon_laser_power', u'moon_laser_duration']

and the data variable would contain a list of points, returned in the same order as the codenames:

[[1414150538.551638, 47.0], [1414150538.551662, 42.0]]

The codename#json command

Note

The codename in the command should be substituted with an actual codename

It is also possible to ask for a single point by name.:

import json
command = 'moon_laser_power#json'
sock.sendto(command, host_port)
data = json.loads(sock.recv(2048))

At which point the data variable would contains just a single point as a list:

[1414151366.400581, 47.0]

The raw_wn, codenames_raw, raw and codename#raw commands

These commands do exactly the same as their json counterparts, only that the data is not encoded as json, but with the homemade raw encoding.

Warning

The raw encoding is manually serialized, which is an 100% guarantied error prone approach, so use the json variant whenever possible. Even Labview® to some extent supports json as of version 2013.

Remember that when receiving data in the raw encoding, it should not be json decoded, so the code to work with these commands will look like this:

command = 'some_raw_command'  # E.g. raw_wn
sock.sendto(command, host_port)
data = sock.recv(2048)

There format if the raw encoding is documented in the API documentation for the PullUDPHandler.handle() method.

Below are a simple list of each type of raw command and example out:

command = 'raw_wn'
# Output
'moon_laser_power:1414154338.17,47.0;moon_laser_duration:1414154338.17,42.0'

command = 'codenames_raw'
# Output
'moon_laser_power,moon_laser_duration'

command = 'raw'
# Output
'1414154433.4,47.0;1414154433.4,42.0'

command = 'moon_laser_power#raw'
# Output
'1414154485.08,47.0'

DataPushSocket, send data (network variable)

To receive data on a machine, the DataPushSocket can be used. To set it up simply to be able to see the last received data and the updated total data, set it up like this:

from PyExpLabSys.common.sockets import DataPushSocket
name = 'Data receive socket for giant laser on the moon'
dps = DataPushSocket(name, action='store_last')
# Get data
timestamp_last, last = dps.last
timestamp_updated, updated = dps.updated
# ... do whatever and stop socket
dps.stop()

After settings this up, the last received data set will be available as a tuple in the DataPushSocket.last property, where the first value is the Unix timestamp of reception and the second value is a dictionary with the last received data (all data sent to the DataPushSocket is in dictionary form, see PushUDPHandler.handle() for details). Alternatively, the DataPushSocket.updated property contains the last value received ever for each key in the dict (i.e. not only in the last transmission).

Command examples

The socket server understands commands in two formats, a json and a raw encoded one. For details about how to send commands to a socket server and receive the reply in the two different encodings, see the sections Client side and Command and data examples from the DateDataPullSocket make data available (network variable) section.

The json commands looks like this:

json_wn#{"greeting": "Live long and prosper", "number": 47}
json_wn#{"number": 42}

After the first command the data values in both the DataPushSocket.last and the DataPushSocket.updated properties are:

{u'greeting': u'Live long and prosper', u'number': 47}

After the second command the value in the DataPushSocket.last property is:

{u'number': 42}

and the value in the DataPushSocket.updated property is:

{u'greeting': u'Live long and prosper', u'number': 42}

The commands can also be raw encoded, in which case the commands above will be:

raw_wn#greeting:str:Live long and prosper;number:int:47
raw_wn#number:int:42

Warning

See the warning about the raw encoding.

Upon receipt, the socket server will make a message available on the socket, that contains a status for the receipt and a copy of the data it has gathered (in simple Python string representation). I will look like this:

ACK#{u'greeting': u'Live long and prosper', u'number': 47}

If it does not understand the data, e.g. if it is handed badly formatted raw data, it will return an error:

Sending: "raw_wn#number:88"
Will return: "ERROR#The data part 'number:88' did not match the expected format of 3 parts divided by ':'"

Sending: "raw_wn#number:floats:88"
Will return: "ERROR#The data type 'floats' is unknown. Only ['int', 'float', 'bool', 'str'] are allowed"

DataPushSocket, see all data sets received (enqueue them)

To receive data and make sure that each and every point is reacted to, it is possible to ask the socket to enqueue the data. It is set up in the following manner:

from PyExpLabSys.common.sockets import DataPushSocket
name = 'Command receive socket for giant laser on the moon'
dps = DataPushSocket(name, action='enqueue')
queue = dps.queue  # Local variable to hold the queue
# Get on point
print queue.get()
# Get all data
for _ in range(queue.qsize()):
    print queue.get()
# ... do whatever and stop socket
dps.stop()

As seen above, the queue that holds the data items, is available as the DataPushSocket.queue attribute. Data can be pulled out by calling get() on the queue. NOTE: The for-loop only gets all the data that was available at the time of calling qsize(), so if the actions inside the for loop takes time, it is possible that new data will be enqueued while the for-loop is running, which it will not pull out.

If it is desired to use an existing queue or to set up a queue with other than default settings, the DataPushSocket can be instantiated with a custom queue.

Command examples

Examples of commands that can be sent are the same as in the code example above, after which the queue would end up containing the two dictionaries:

{u'greeting': u'Live long and prosper', u'number': 47}
{u'number': 42}

DataPushSocket, make socket call function on reception (callback)

With the DataPushSocket it is also possible to ask the socket to call a callback function on data reception:

from PyExpLabSys.common.sockets import DataPushSocket
import time

# Module variables
STATE = 'idle'
STOP = False

def callback_func(data):
    """Callback function for the socket"""
    print 'Received: {}'.format(data)
    #... do fancy stuff depending on the data, e.g. adjust laser settings
    # or fire (may change STATE)

name = 'Command callback socket for giant laser on the moon'
dps = DataPushSocket(name, action='callback_async', callback=callback_func)

while not STOP:
    # Check if there is a need for continuous activity e.g. monitor
    # temperature of giant laser during usage
    if STATE == 'fire':
        # This function should end when STATE changes away from 'fire'
        monitor_temperature()
    time.sleep(1)

# After we are all done, stop the socket
dps.stop()

In this examples, the data for the callbacks (and therefore the callbacks themselves) will be queued up and happen asynchronously. This makes it possible to send a batch of commands without waiting, but there is no monitoring of whether the queue is filled faster than it can be emptied. It can of course be checked by the user, but if there is a need for functionality in which the sockets make such checks itself and rejects data if there is too much in queue, then talk to the development team about it.

Command examples

Command examples this kind of socket will as always be dicts, but in this case will likely have to contain some information about which action to perform or method to call, but that is entirely up to the user, since that is implemented by the user in the call back function. Some examples could be:

json_wn#{"action": "fire", "duration": 8, "intensity": 300}
json_wn#{"method": "fire", "duration": 8, "intensity": 300}

DataPushSocket, control class and send return values back (callback with return)

This is reduced version of an example that shows two things:

• How to get the return value when calling a function via the DataPushSocket
• How to control an entire class with a DataPushSocket

from PyExpLabSys.common.sockets import DataPushSocket

class LaserControl(object):
    """Class that controls the giant laser laser on the moon"""

    def __init__(self):
        self.settings = {'power': 100, 'focus': 100, 'target': None}
        self._state = 'idle'

        # Start socket
        name = 'Laser control, callback socket, for giant laser on the moon'
        self.dps = DataPushSocket(name, action='callback_direct',
                                  callback=self.callback)
        self.dps.start()

        self.stop = False
        # Assume one of the methods can set stop
        while not self.stop:
            # Do continuous stuff on command
            time.sleep(1)
        self.dps.stop()

    def callback(self, data):
        """Callback and central control function"""
        # Get the method name and don't pass it on as an argument
        method_name = data.pop('method')
        # Get the method
        method = self.__getattribute__(method_name)
        # Call the method and return its return value
        return method(**data)

    def update_settings(self, **kwargs):
        """Update settings"""
        for key in kwargs.keys():
            if key not in self.settings.keys():
                message = 'Not settings for key: {}'.format(key)
                raise ValueError(message)
        self.settings.update(kwargs)
        return 'Updated settings with: {}'.format(kwargs)

    def state(self, state):
        """Set state"""
        self._state = state
        return 'State set to: {}'.format(state)

This socket would then be sent commands in the form of json encoded dicts from an UDP client in the secret lair. These dicts could look like:

{'method': 'update_settings', 'power': 300, 'focus': 10}
# or
{'method': 'state', 'state': 'active'}

which would, respectively, make the update_settings method be called with the arguments power=300, focus=10 and the state method be called with the argument state='active'. NOTE: In this implementation, it is the responsibility of the caller that the method name exists and that the arguments that are sent have the correct names. An alternative, but less flexible, way to do the same, would be to make an if-elif-else structure on the method name and format the arguments in manually:

def callback(self, data):
    """Callback and central control function"""
    method_name = data.get('method')
    if method_name == 'state':
        if data.get('state') is None:
            raise ValueError('Argument \'state\' missing')
        out = self.state(data['state'])
    elif method_name == 'update_settings':
        # ....
        pass
    else:
        raise ValueError('Unknown method: {}'.format(method_name))

    return out

The complete and running example of both server and client for this example can be downloaded in these two files: server, client.

Command examples

See the attached files with example code for command examples.

Port defaults

To make for easier configuration on both ends of the network communication, the different kinds of sockets each have their own default port. They are as follows:

Socket	Default port
DateDataPullSocket	9000
DataPullSocket	9010
DataPushSocket	8500
LiveSocket	8000

Again, to ease configuration also on the client side, if more than one socket of the same kind is needed on one machine, then it is recommended to simply add 1 to the port number for each additional socket.

Inheritance

The DateDataPullSocket and DataPullSocket classes inherit common functionality, such as;

• Input checks
• Initialization of DATA
• Methods to start and stop the thread and reset DATA

from the CommonDataPullSocket class, as illustrated in the diagram below.

digraph inheritance65e5d72deb { rankdir=LR; size="8.0, 12.0"; "PyExpLabSys.common.sockets.CallBackThread" [URL="#PyExpLabSys.common.sockets.CallBackThread",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Class to handle the calling back for a DataReceiveSocket"]; "threading.Thread" -> "PyExpLabSys.common.sockets.CallBackThread" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.CommonDataPullSocket" [URL="#PyExpLabSys.common.sockets.CommonDataPullSocket",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Abstract class that implements common data pull socket functionality."]; "threading.Thread" -> "PyExpLabSys.common.sockets.CommonDataPullSocket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.DataPullSocket" [URL="#PyExpLabSys.common.sockets.DataPullSocket",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="This class implements a UDP socket server for serving x, y type data."]; "PyExpLabSys.common.sockets.CommonDataPullSocket" -> "PyExpLabSys.common.sockets.DataPullSocket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.DataPushSocket" [URL="#PyExpLabSys.common.sockets.DataPushSocket",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="This class implements a data push socket and provides options for"]; "threading.Thread" -> "PyExpLabSys.common.sockets.DataPushSocket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.DateDataPullSocket" [URL="#PyExpLabSys.common.sockets.DateDataPullSocket",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="This class implements a UDP socket server for serving data as a function"]; "PyExpLabSys.common.sockets.CommonDataPullSocket" -> "PyExpLabSys.common.sockets.DateDataPullSocket" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.LiveSocket" [URL="#PyExpLabSys.common.sockets.LiveSocket",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="This class implements a Live Socket"]; "PyExpLabSys.common.sockets.PortStillReserved" [URL="#PyExpLabSys.common.sockets.PortStillReserved",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Custom exception to explain socket server port still reserved even after"]; "PyExpLabSys.common.sockets.PullUDPHandler" [URL="#PyExpLabSys.common.sockets.PullUDPHandler",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Request handler for the :class:`.DateDataPullSocket` and"]; "socketserver.BaseRequestHandler" -> "PyExpLabSys.common.sockets.PullUDPHandler" [arrowsize=0.5,style="setlinewidth(0.5)"]; "PyExpLabSys.common.sockets.PushUDPHandler" [URL="#PyExpLabSys.common.sockets.PushUDPHandler",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="This class handles the UDP requests for the :class:`.DataPushSocket`"]; "socketserver.BaseRequestHandler" -> "PyExpLabSys.common.sockets.PushUDPHandler" [arrowsize=0.5,style="setlinewidth(0.5)"]; "socketserver.BaseRequestHandler" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="Base class for request handler classes."]; "threading.Thread" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A class that represents a thread of control."]; }

Status of a socket server

All 4 socket servers understand the status command. This command will return some information about the status of the machine the socket server is running on and the status of all socket servers running on this machine. The reason the command returns the status for all the socket servers running on the machine is, that what this command is really meant for, is to get the status of the system and so it should not be necessary to communicate with several socket servers to get that.

The data returned is a json encoded dictionary, which looks like this:

{u'socket_server_status':
          {u'8000': {u'name': u'my_live_socket',
                     u'since_last_activity': 0.0009279251098632812,
                     u'status': u'OK',
                     u'type': u'live'},
           u'9000': {u'name': u'my_socket',
                     u'since_last_activity': 0.0011229515075683594,
                     u'status': u'OK',
                     u'type': u'date'}},
 u'system_status':
          {u'filesystem_usage': {u'free_bytes': 279182213120,
                                 u'total_bytes': 309502345216},
           u'last_apt_cache_change_unixtime': 1413984912.529932,
           u'last_git_fetch_unixtime': 1413978995.4109764,
           u'load_average': {u'15m': 0.14, u'1m': 0.1, u'5m': 0.15},
           u'max_python_mem_usage_bytes': 37552128,
           u'number_of_python_threads': 3,
           u'uptime': {u'idle_sec': 321665.77,
                       u'uptime_sec': 190733.39}}}

Socket server status

The socket servers status is broken down into one dictionary for each socket server, indexed by their ports. The status for the individual socket server comprise of the following items:

name (str):The name of the socket server since_last_activity (float):  The number of seconds since last activity on the socket server status (str):The status of the socket server. It will return either 'OK' if the last activity was newer than the activity timeout, or 'INACTIVE' if the last activity was older than the activity timeout or 'DISABLED' is activity monitoring is disabled for the socket server. type:The type of the socket server

System status

The system status items depends on the operating system the socket server is running on.

All operating systems

last_git_fetch_unixtime (float):  The Unix timestamp of the last git fetch of the ‘origin’ remote, which points at the Github archive max_python_mem_usage_bytes (int):  The maximum memory usage of Python in bytes number_of_python_threads (int):  The number of Python threads in use

Linux operating systems

uptime (dict):Information about system uptime (from /proc/uptime). The value 'uptime_sec' contains the system uptime in seconds and the value 'idle_sec' contains the idle time in seconds. NOTE: While the uptime is measured in wall time, the idle time is measured in CPU time, which means that if the system is multi-core, it will add up idle time for all the cores. last_apt_cache_change_unixtime (float):  The Unix time stamp of the last change to the apt cache, which should be a fair approximation to the last time the system was updated load_average(dict):  The load average (roughly number of active processes) over the last 1, 5 and 15 minutes (from /proc/loadavg). For a detailed explanation see the /proc/loadavg section from the proc man-page filesystem_usage (dict):  The number of total and free bytes for the file-system the PyExpLabSys archive is located on

Auto-generated module documentation

The sockets module contains various implementations of UDP socket servers (at present 4 in total) for transmission of data over the network. The different implementations are tailored for a specific purposes, as described below.

In general, there is a distinction in the naming of the different socket server implementation between push socket servers, that you can push data to, and pull socket servers, that you can pull data from.

Presently the module contains the following socket servers:

• DateDataPullSocket (DateDataPullSocket) This socket server is used to make continuous data (i.e. one-value data as function of time) available on the network. It identifies different data channels by codenames and has a timeout functionality to prevent serving old data to the client.
• DataPullSocket (DataPullSocket) This socket is similar to the date data server, but is used to make x, y type data available on the network. It identifies different data channel by codenames and has timeout functionality to prevent serving old data.
• DataPushSocket (DataPushSocket) This socket is used to recieve data from the network. The data is received in dictionary form and it identifies the data channels by codenames (keys) in the dictionaries. It will save the last point, and the last value for each codename. It also has the option to, for each received data set, to put them in a queue (that the user can then empty) or to call a callback function with the received data as en argument.
• LiveSocket (LiveSocket) This socket is used only for serving data to the live socket server. It also is not actually a socket server like the others, but it has a similar interface.

Note

The module variable DATA is a dict shared for all socket servers started from this module. It contains all the data, queues, settings etc. It can be a good place to look if, to get a behind the scenes look at what is happening.

PyExpLabSys.common.sockets.bool_translate(string)

Returns boolean value from strings ‘True’ or ‘False’

PyExpLabSys.common.sockets.socket_server_status()

Returns the status of all socket servers

Returns:

Dict with port to status dict mapping. The status dict has the following keys:

name, type, status (with str values) and since_last_activity with float value.

Return type:dict

class PyExpLabSys.common.sockets.PullUDPHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

Request handler for the DateDataPullSocket and DateDataPullSocket socket servers. The commands this request handler understands are documented in the handle() method.

handle()

Returns data corresponding to the request

The handler understands the following commands:

COMMANDS

• raw (str): Returns all values on the form x1,y1;x2,y2 in the order the codenames was given to the DateDataPullSocket.__init__() or DataPullSocket.__init__() method
• json (str): Return all values as a list of points (which in themselves are lists) e.g: [[x1, y1], [x2, y2]]), contained in a json string. The order is the same as in raw.
• raw_wn (str): (wn = with names) Same as raw but with names, e.g. codenam1:x1,y1;codename2:x2,y2. The order is the same as in raw.
• json_wn (str): (wn = with names) Return all data as a dict contained in a json string. In the dict the keys are the codenames.
• codename#raw (str): Return the value for codename on the form x,y
• codename#json (str): Return the value for codename as a list (e.g [x1, y1]) contained in a json string
• codenames_raw (str): Return the list of codenames on the form name1,name2
• codenames_json (str): Return a list of the codenames contained in a json string
• name (str): Return the name of the socket server
• status (str): Return the system status and status for all socket servers.

class PyExpLabSys.common.sockets.CommonDataPullSocket(name, codenames, port, default_x, default_y, timeouts, check_activity, activity_timeout, init_timeouts=True, handler_class=<class 'PyExpLabSys.common.sockets.PullUDPHandler'>)

Bases: threading.Thread

Abstract class that implements common data pull socket functionality.

This common class is responsible for:

• Initializing the thread
• Checking the inputs
• Starting the socket server with the correct handler
• Initializing DATA with common attributes

__init__(name, codenames, port, default_x, default_y, timeouts, check_activity, activity_timeout, init_timeouts=True, handler_class=<class 'PyExpLabSys.common.sockets.PullUDPHandler'>)

Initializes internal variables and data structure in the DATA module variable

Parameters:

• name (str) – The name of the DataPullSocket server. Used for identification and therefore should contain enough information about location and purpose to unambiguously identify the socket server. E.g: 'DataPullSocket with data from giant laser on the moon'
• codenames (list) – List of codenames for the measurements. The names must be unique and cannot contain the characters: #,;: and SPACE
• port (int) – Network port to use for the socket (deafult 9010)
• default_x (float) – The x value the measurements are initiated with
• default_y (float) – The y value the measurements are initiated with
• timeouts (float or list of floats) – The timeouts (in seconds as floats) the determines when the date data socket regards the data as being to old and reports that. If a list of timeouts is supplied there must be one value for each codename and in the same order.
• init_timeouts (bool) – Whether timeouts should be instantiated in the DATA module variable
• handler_class (Sub-class of SocketServer.BaseRequestHandler) – The UDP handler to use in the server
• check_activity (bool) – Whether the socket server should monitor activity. What detemines activity is described in the derived socket servers.
• activity_timeout (float or int) – The timespan in seconds which constitutes in-activity

run()

Starts the UPD socket server

stop()

Stops the UDP server

Note

Closing the server and deleting the socket server instance is necessary to free up the port for other usage

poke()

Pokes the socket server to let it know that there is activity

class PyExpLabSys.common.sockets.DataPullSocket(name, codenames, port=9010, default_x=0.0, default_y=0.0, timeouts=None, check_activity=True, activity_timeout=900, poke_on_set=True)

Bases: PyExpLabSys.common.sockets.CommonDataPullSocket

This class implements a UDP socket server for serving x, y type data. The UDP server uses the PullUDPHandler class to handle the UDP requests. The commands that can be used with this socket server are documented in the PullUDPHandler.handle() method.

__init__(name, codenames, port=9010, default_x=0.0, default_y=0.0, timeouts=None, check_activity=True, activity_timeout=900, poke_on_set=True)

Initializes internal variables and UPD server

For parameter description of name, codenames, port, default_x, default_y, timeouts, check_activity and activity_timeout see CommonDataPullSocket.__init__().

Parameters:poke_on_set (bool) – Whether to poke the socket server when a point is set, to let it know there is activity

set_point(codename, point, timestamp=None)

Sets the current point for codename

Parameters:

• codename (str) – Name for the measurement whose current point should be set
• value (iterable) – Current point as a list or tuple of 2 floats: [x, y]
• timestamp (float) – A unix timestamp that indicates when the point was measured. If it is not set, it is assumed to be now. This value is used to evaluate if the point is new enough if timeouts are set.

class PyExpLabSys.common.sockets.DateDataPullSocket(name, codenames, port=9000, default_x=0.0, default_y=0.0, timeouts=None, check_activity=True, activity_timeout=900, poke_on_set=True)

Bases: PyExpLabSys.common.sockets.CommonDataPullSocket

This class implements a UDP socket server for serving data as a function of time. The UDP server uses the PullUDPHandler class to handle the UDP requests. The commands that can be used with this socket server are documented in the PullUDPHandler.handle() method.

__init__(name, codenames, port=9000, default_x=0.0, default_y=0.0, timeouts=None, check_activity=True, activity_timeout=900, poke_on_set=True)

Init internal variavles and UPD server

For parameter description of name, codenames, port, default_x, default_y, timeouts, check_activity and activity_timeout see CommonDataPullSocket.__init__().

Parameters:poke_on_set (bool) – Whether to poke the socket server when a point is set, to let it know there is activity

set_point_now(codename, value)

Sets the current y-value for codename using the current time as x

Parameters:

• codename (str) – Name for the measurement whose current value should be set
• value (float) – y-value

set_point(codename, point)

Sets the current point for codename

Parameters:

• codename (str) – Name for the measurement whose current point should be set
• point (iterable) – Current point as a list (or tuple) of 2 floats: [x, y]

class PyExpLabSys.common.sockets.PushUDPHandler(request, client_address, server)

Bases: socketserver.BaseRequestHandler

This class handles the UDP requests for the DataPushSocket

handle()

Sets data corresponding to the request

The handler understands the following commands:

COMMANDS

• json_wn#data (str): Json with names. The data should be a JSON encoded dict with codename->value content. A complete command could look like:

  'json_wn#{"greeting": "Live long and prosper", "number": 47}'

• raw_wn#data (str): Raw with names. The data should be a string with data on the following format: codename1:type:data;codename2:type:data;... where type is the type of the data and can be one of 'int', 'float', 'str' and 'bool'. NOTE; that neither the names or any data strings can contain any of the characters in BAD_CHARS. The data is a comma separated list of data items of that type. If there is more than one, they will be put in a list. An example of a complete raw_wn string could look like:

  'raw_wn#greeting:str:Live long and prosper;numbers:int:47,42'

• name (str): Return the name of the PushSocket server

• status (str): Return the system status and status for all socket servers.

• commands (str): Return a json encoded list of commands. The returns value is is prefixed with PUSH_RET and ‘#’ so e.g. ‘RET#actual_date’

class PyExpLabSys.common.sockets.DataPushSocket(name, port=8500, action='store_last', queue=None, callback=None, return_format='json', check_activity=False, activity_timeout=900)

Bases: threading.Thread

This class implements a data push socket and provides options for enqueuing, calling back or doing nothing on reciept of data

__init__(name, port=8500, action='store_last', queue=None, callback=None, return_format='json', check_activity=False, activity_timeout=900)

Initializes the DataPushSocket

Parameters:

• name (str) – The name of the socket server. Used for identification and therefore should contain enough information about location and purpose to unambiguously identify the socket server. E.g: 'Driver push socket for giant laser on the moon'
• port (int) – The network port to start the socket server on (default is 8500)
• action (string) –

  Determined the action performed on incoming data. The possible values are:

  • 'store_last' (default and always) the incoming data will be stored, as a dict, only in the two properties; last and updated, where last contains only the data from the last reception and updated contains the newest value for each codename that has been received ever. Saving to these two properties will always be done, also with the other actions.
  • 'enqueue'; the incoming data will also be enqueued
  • 'callback_async' a callback function will also be called with the incoming data as an argument. The calls to the callback function will in this case happen asynchronously in a seperate thread
  • 'callback_direct' a callback function will also be called and the result will be returned, provided it has a str representation. The return value format can be set with return_format

• queue (Queue.Queue) – If action is ‘enqueue’ and this value is set, it will be used as the data queue instead the default which is a new Queue.Queue instance without any further configuration.
• callback (callable) – A callable that will be called on incoming data. The callable should accept a single argument that is the data as a dictionary.
• return_format (str) –

  The return format used when sending callback return values back (used with the 'callback_direct' action). The value can be:

  • 'json', which, if possible, will send the value back encoded as json
  • 'raw' which, if possible, will encode a dict of values, a list of lists or None. If it is a dict, each value may be a list of values with same type, in the same way as they are received with the 'raw_wn' command in the PushUDPHandler.handle() method. If the return value is a list of lists (useful e.g. for several data points), then all values must be of the same type. The format sent back looks like: '1.0,42.0&1.5,45.6&2.0,47.0', where ‘&’ separates the inner lists and ‘,’ the points in those lists
  • 'string' in which the string representation of the value the call back returns will be sent back. NOTE: These string representations may differ between Python 2 and 3, so do not parse them

run()

Starts the UPD socket server

stop()

Stops the UDP socket server

Note

Closing the server and deleting the SocketServer.UDPServer socket instance is necessary to free up the port for other usage

queue

Gets the queue, returns None if action is 'store_last' or 'callback_direct'

last

Gets a copy of the last data

Returns

tuple: (last_data_time, last_data) where last_data is the

data from the last reception and last_data_time is the Unix timestamp of that reception. Returns (None, None) if no data has been recieved.

updated

Gets a copy of the updated total data, returns empty dict if no data has been received yet

Returns:

(updated_data_time, updated_data) where updated_data

is the total updated data after the last reception and updated_data_time is the Unix timestamp of that reception. Returns (None, {}) if no data has been recieved.

Return type:tuple

set_last_to_none()

Sets the last data point and last data point time to None

clear_updated()

Clears the total updated data and set the time of last update to None

poke()

Pokes the socket server to let it know that there is activity

class PyExpLabSys.common.sockets.CallBackThread(queue, callback)

Bases: threading.Thread

Class to handle the calling back for a DataReceiveSocket

__init__(queue, callback)

Initialize the local variables

Parameters:

• queue (Queue.Queue) – The queue that queues up the arguments for the callback function
• callback (callable) – The callable that will be called when there are items in the queue

run()

Starts the calling back

stop()

Stops the calling back

exception PyExpLabSys.common.sockets.PortStillReserved

Bases: Exception

Custom exception to explain socket server port still reserved even after closing the port

__init__()

Initialize self. See help(type(self)) for accurate signature.

class PyExpLabSys.common.sockets.LiveSocket(name, codenames, live_server=None, no_internal_data_pull_socket=False, internal_data_pull_socket_port=8000)

Bases: object

This class implements a Live Socket

As of version 2 LiveSocket there are a few new features to note:

1. There is now support for values of any json-able object. The values can of course only be shown in a graph if they are numbers, but all other types can be shown in a table.
2. There is now support for generic xy data. Simply use set_point() or set_batch() and give it x, y values.

__init__(name, codenames, live_server=None, no_internal_data_pull_socket=False, internal_data_pull_socket_port=8000)

Intialize the LiveSocket

Parameters:

• name (str) – The name of the socket
• codenames (sequence) – The codenames for the different data channels on this LiveSocket
• live_server (sequence) – 2 element sequence of hostname and port for the live server to connect to. Defaults to (Settings.common_liveserver_host, Settings.common_liveserver_host).
• no_internal_data_pull_socket (bool) – Whether to not open an internal DataPullSocket. Defaults to False. See note below.
• internal_data_pull_socket_port (int) – Port for the internal DataPullSocket. Defaults to 8000. See note below.

Note

In general, any socket should also work as a status socket. But since the new design of the live socket, it no longers runs a UDP server, as would be required for it to work as a status socket. Therefore, LiveSocket now internally runs a DataPullSocket on port 8000 (that was the old LiveSocket port) to work as a status socket. With default setting, everything should work as before.

start()

Starts the internal DataPullSocket

stop()

Stop the internal DataPullSocket

set_batch(data)

Set a batch of points now

Parameters:data (dict) – Batch of data on the form {codename1: (x1, y1), codename2: (x2, y2)}. Note, that for the live socket system, the y values need not be data in the form of a float, it can also be an int, bool or str. This is done to make it possible to also transmit e.g. equipment status to the live pages.

Note

All data is sent to the live socket proxy and onwards to the web browser clients as batches, so if the data is on batch form, might as well send it as such and reduce the number of transmissions.

set_batch_now(data)

Set a batch of point now

Parameters:data (dict) – A mapping of codenames to values without times or x-values (see example below)

The format for data is:

{'measurement1': 47.0, 'measurement2': 42.0}

set_point_now(codename, value)

Sets the current value for codename using the current time as x

Parameters:

• codename (str) – Name for the measurement whose current value should be set
• value (float, int, bool or str) – value

set_point(codename, point)

Sets the current point for codename

Parameters:

• codename (str) – Name for the measurement whose current point should be set
• point (list or tuple) – Current value “point” as a list (or tuple) of items, the first must be a float, the second can be float, int, bool or str

reset(codenames)

Send the reset signal for codenames

Parameters:codenames (list) – List of codenames

PyExpLabSys.common.sockets.BAD_CHARS = ['#', ',', ';', ':', '&']

The list of characters that are not allowed in code names

PyExpLabSys.common.sockets.UNKNOWN_COMMAND = 'UNKNOWN_COMMMAND'

The string returned if an unknown command is sent to the socket

PyExpLabSys.common.sockets.OLD_DATA = 'OLD_DATA'

The string used to indicate old or obsoleted data

PyExpLabSys.common.sockets.PUSH_ERROR = 'ERROR'

The answer prefix used when a push failed

PyExpLabSys.common.sockets.PUSH_ACK = 'ACK'

The answer prefix used when a push succeds

PyExpLabSys.common.sockets.PUSH_EXCEP = 'EXCEP'

The answer prefix for when a callback or callback value formatting produces an exception

PyExpLabSys.common.sockets.PUSH_RET = 'RET'

The answer prefix for a callback return value

PyExpLabSys.common.sockets.DATA = {}

The variable used to contain all the data.

The format of the DATA variable is the following. The DATA variable is a dict, where each key is an integer port number and the value is the data for the socket server on that port. The data for each individual socket server is always a dict, but the contained values will depend on which kind of socket server it is, Examples below.

For a DateDataPullSocket the dict will resemble this example:

{'activity': {'activity_timeout': 900,
              'check_activity': True,
              'last_activity': 1413983209.82526},
 'codenames': ['var1'],
 'data': {'var1': (0.0, 0.0)},
 'name': 'my_socket',
 'timeouts': {'var1': None},
 'type': 'date'}

For a DataPullSocket the dict will resemble this example:

{'activity': {'activity_timeout': 900,
              'check_activity': True,
              'last_activity': 1413983209.825451},
 'codenames': ['var1'],
 'data': {'var1': (0.0, 0.0)},
 'name': 'my_data_socket',
 'timeouts': {'var1': None},
 'timestamps': {'var1': 0.0},
 'type': 'data'}

For a DataPushSocket the dict will resemble this example:

{'action': 'store_last',
 'activity': {'activity_timeout': 900,
              'check_activity': False,
              'last_activity': 1413983209.825681},
 'last': None,
 'last_time': None,
 'name': 'my_push_socket',
 'type': 'push',
 'updated': {},
 'updated_time': None}

PyExpLabSys.common.sockets.TYPE_FROM_STRING = {'bool': <function bool_translate>, 'float': <class 'float'>, 'int': <class 'int'>, 'str': <class 'str'>}

The dict that transforms strings to convertion functions

PyExpLabSys.common.sockets.run_module()

This functions sets