Binary namedtuples in Python
Many systems have a need to efficiently manage a lot of boolean values associated with users or other objects. Just think about all of those different notification settings many applications offer. Services that use OAuth2 to control access to their APIs may have a lot of different scopes that limit access and refresh tokens. Some systems may also have fine grained permissions associated with different user accounts. All of these scenarios can require these values to be readily available for active users or API clients. You'd like to be able to efficiently cache/store these things but also easily work with them at runtime. Many approaches often use bitwise operators or bit masks to track these settings but that can often lead to code nobody wants to touch as the number of values grows large. A container datatype often overlooked for things like this in Python is the namedtuple:
"Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples."
— from https://docs.python.org/3.4/library/collections.html#collections.namedtuple
A named tuple is a regular Python class, so you can easily subclass it and attach additional functionality. So we can have a named tuple that allows us to easily access boolean settings/values via attributes that also knows how to represent itself in a binary form. This makes it easy to work with in code and also allows for efficient caching along with a user session or API access token. Consider this simple example:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
from collections import namedtuple #Just add new attribute names to the end of this list attribute_names = ['is_admin', 'email_notifications', 'push_notifications', 'delete_object', 'read_object'] BaseNamedTuple = namedtuple('BinaryNamedTuple', attribute_names) class BinaryNamedTuple(BaseNamedTuple): """ The namedtuple you love filled with boolean values and including binary export/import. Can be used to represent different settings, permissions, OAuth2 scope, whatever your want... Useful anytime you've got a lot of boolean values to track for an object, user, etc. """ def __new__(cls, attributes=None): """ Create a new BinaryNamedTuple based on the specified list of attributes :param cls: BinaryNamedTuple :param attributes: a list of attributes that should be set to True :return: a new BinaryNamedTuple with the appropriate attributes set to True """ if attributes is None: attributes = [] attribute_values = [True if attribute in attributes else False for attribute in attribute_names] return super(BaseNamedTuple, cls).__new__(cls, attribute_values) @classmethod def from_binary(cls, namedtuple_bits): """ Converts a binary representation of BinaryNamedTuple back to a BinaryNamedTuple :param namedtuple_bits: an integer (binary representation of BinaryNamedTuple) :return: a new BinaryNamedTuple initialized based on namedtuple_bits """ attribute_values = [] bit = 1 for attribute in attribute_names: if namedtuple_bits & bit != 0: attribute_values.append(attribute) bit <<= 1 return cls(attribute_values) def to_binary(self): """ Converts a BinaryNamedTuple to a binary representation :return: an integer (binary representation of BinaryNamedTuple) """ bits = 0 for index, attribute in enumerate(attribute_names): if getattr(self, attribute, False): bits += 1 << index return bits |
A new BinaryNamedTuple can be initialized with a list of attributes that should be set to True. All other defined attributes will be initialized as False. If using BinaryNamedTuple within an OAuth2 implementation, this might be the list of granted scope names. The named tuple also provides a nice repr when working with them interactively or during debugging sessions.
|
1 2 3 4 5 |
>>> settings = BinaryNamedTuple(['delete_object', 'push_notifications']) >>> settings BinaryNamedTuple(is_admin=False, email_notifications=False, push_notifications=True, delete_object=True, read_object=False) |
Anytime settings need to be updated and cached, just make a new BinaryNamedTuple and use the to_binary method to store a binary representation of it.
|
1 2 |
>>> settings.to_binary() 12 |
When you need to switch back to named tuple form, just use from_binary to quickly get back to a more manageable form:
|
1 2 3 4 5 |
>>>settings = BinaryNamedTuple.from_binary(12) >>>settings BinaryNamedTuple(is_admin=False, email_notifications=False, push_notifications=True, delete_object=True, read_object=False) |
Attribute values are easily accessed and can be used during a web/API request cycle as needed to enforce rules, data access requirements or pretty much anything else. They're right at home inside a decorator protecting your Flask views.
|
1 2 |
>>> settings.push_notifications True |
|
1 2 |
>>> settings.is_admin False |
Another nice benefit of using named tuples is that, like tuples, they're immutable. So you can avoid accidents where values might be changed outside of the normal process and bugs are introduced.
|
1 2 3 4 5 6 7 8 |
>>> settings.push_notifications = True --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in () ----> 1 settings.push_notifications = True AttributeError: can't set attribute |
...and just in case you were wondering about that part where “named tuples don't take up any more space than a regular tuple”:
|
1 2 3 |
>>> import sys >>> sys.getsizeof(settings) 96 |
|
1 2 |
>>> sys.getsizeof((False, False, True, True, False)) 96 |
The next time you're faced with efficiently managing a lot of boolean values for user settings or an API token, don't forget about your friend the (binary) namedtuple.
- The gremlin-python library - February 24, 2015
- Binary namedtuples in Python - January 14, 2015
- Apply Out-of-Pocket Health Costs to Deductibles in One Click - October 14, 2014
- Business Managers - November 18, 2013
- Shirt, Shoes, and JSON Required - October 2, 2012
Tags: Dev
Peter
April 23, 2018 at 12:00 am
Nicely done, thank you for sharing, this has come in useful!