Welcome to aiogibson’s documentation!

aiogibson is a library for accessing a gibson cache database from the asyncio (PEP-3156/tulip) framework.

Gibson is a high efficiency, tree based memory cache server. It uses a special trie structure allowing the user to perform operations on multiple key sets using a prefix expression achieving the same performance grades in the worst case, even better on an average case then regular cache implementations based on hash tables.

Contents:

Installation

The easiest way to install aiogibson is by using the package on PyPi:

pip install aiogibson

Contribute

Feel free to file an issue or make pull request if you find any bugs or have some suggestions for library improvement.

Contents

Getting started

The easiest way to install aiogibson is by using the package on PyPi:

pip install aiogibson

Make sure that gibson server installed and started according official documentation. We assume that you have your gibson started using unix sockets (by default) with address /tmp/gibson.sock, your python version >= 3.3.

aiogibson has straightforward api, just like memcached:

Basic Example

import asyncio
from aiogibson import create_gibson

loop = asyncio.get_event_loop()


@asyncio.coroutine
def go():
    gibson = yield from create_gibson('/tmp/gibson.sock', loop=loop)
    # set value
    yield from gibson.set(b'foo', b'bar', 7)
    yield from gibson.set(b'numfoo', 100, 7)

    # get value
    result = yield from gibson.get(b'foo')
    print(result)

    # set ttl to the value
    yield from gibson.ttl(b'foo', 10)

    # increment given key
    yield from gibson.inc(b'numfoo')

    # decrement given key
    yield from gibson.dec(b'numfoo')

    # lock key from modification
    yield from gibson.lock(b'numfoo')

    # unlock given key
    yield from gibson.unlock(b'numfoo')

    # delete value
    yield from gibson.delete(b'foo')

    # Get system stats about the Gibson instance
    info = yield from gibson.stats()


loop.run_until_complete(go())

Underlying data structure trie allows us to perform operations on multiple key sets using a prefix expression:

Multi Commands

import asyncio
from aiogibson import create_gibson

loop = asyncio.get_event_loop()


@asyncio.coroutine
def go():
    gibson = yield from create_gibson('/tmp/gibson.sock', loop=loop)

    # set the value for keys verifying the given prefix
    yield from gibson.mset(b'fo', b'bar', 7)
    yield from gibson.mset(b'numfo', 100, 7)

    # get the values for keys with given prefix
    result = yield from gibson.mget(b'fo')

    # set the TTL for keys verifying the given prefix
    yield from gibson.mttl(b'fo', 10)

    # increment by one keys verifying the given prefix.
    yield from gibson.minc(b'numfo')

    # decrement by one keys verifying the given prefix
    yield from gibson.mdec(b'numfoo')

    # lock keys with prefix from modification
    yield from gibson.mlock(b'fo')

    # unlock keys with given prefix
    yield from gibson.munlock(b'fo')

    # delete keys verifying the given prefix.
    yield from gibson.mdelete(b'fo')

    # return list of keys with given prefix ``fo``
    yield from gibson.keys(b'fo')

    # count items for a given prefi
    info = yield from gibson.stats()


loop.run_until_complete(go())

aiogibson has connection pooling support using context-manager:

Connection Pool Example

import asyncio
from aiogibson import create_pool

loop = asyncio.get_event_loop()

@asyncio.coroutine
def go():
    pool = yield from create_pool('/tmp/gibson.sock', minsize=5, maxsize=10,
                                  loop=loop)

    with (yield from pool) as gibson:
        yield from gibson.set('foo', 'bar')
        value = yield from gibson.get('foo')
        print(value)

    pool.clear()

loop.run_until_complete(go())

Also you can have simple low-level interface to gibson server:

Low Level Commands

import asyncio
from aiogibson import create_gibson

loop = asyncio.get_event_loop()


@asyncio.coroutine
def go():
    gibson = yield from create_connection('/tmp/gibson.sock', loop=loop)
    # set value
    yield from gibson.execute(b'set', b'foo', b'bar', 7)
    # get value
    result = yield from gibson.execute(b'get', b'foo')
    print(result)
    # delete value
    yield from gibson.execute(b'del', b'foo')

loop.run_until_complete(go())

aiogibson — API Reference

Connection Object

Low level connection with raw interface.

aiogibson.connection.create_connection(address, *, encoding=None, loop=None)[source]

Creates GibsonConnection connection. Opens connection to Gibson server specified by address argument.

Parameters:
  • address – str for unix socket path, or tuple for (host, port) tcp connection.
  • encoding – this argument can be used to decode byte-replies to strings. By default no decoding is done.
class aiogibson.connection.GibsonConnection(reader, writer, address, *, encoding=None, loop=None)[source]

Gibson connection.

close()[source]

Close connection.

closed[source]

True if connection is closed.

encoding[source]

Current set codec or None.

execute(command, *args, encoding=<object object at 0x7f4792734560>)[source]

Executes raw gibson command.

Parameters:
  • command – str or bytes gibson command.
  • args – tuple of arguments required for gibson command.
  • encoding – str default encoding for unpacked data.
Raises:
  • TypeError – if any of args can not be encoded as bytes.
  • ProtocolError – when response can not be decoded meaning connection is broken.

Connection Pool

Pool of connection using context manager protocol:

import asyncio
from aiogibson import create_pool

loop = asyncio.get_event_loop()

@asyncio.coroutine
def go():
    pool = yield from create_pool('/tmp/gibson.sock', minsize=5,
                                  maxsize=10, loop=loop)

    with (yield from pool) as gibson:
        yield from gibson.set('foo', 'bar')
        value = yield from gibson.get('foo')
        print(value)

    # or without context manager
    yield from pool.set('foo', 'bar')
    resp = yield from pool.get('foo')
    yield from pool.delete('foo')

    pool.clear()

loop.run_until_complete(go())
aiogibson.pool.create_pool(address, *, encoding=None, minsize=10, maxsize=10, commands_factory=<class 'aiogibson.commands.Gibson'>, loop=None)[source]

Creates Gibson Pool.

By default it creates pool of commands_factory instances, but it is also possible to create pool of plain connections by passing lambda conn: conn as commands_factory. All arguments are the same as for create_connection. Returns GibsonPool instance.

class aiogibson.pool.GibsonPool(address, encoding=None, *, minsize, maxsize, commands_factory, loop=None)[source]

Gibson connections pool.

acquire()[source]

Acquires a connection from free pool.

Creates new connection if needed.

clear()[source]

Clear pool connections.

Close and remove all free connections.

encoding[source]

Current set codec or None.

freesize[source]

Current number of free connections.

maxsize[source]

Maximum pool size.

minsize[source]

Minimum pool size.

release(conn)[source]

Returns connection back into pool. Since this method is used in __exit__(), this method must not be coroutine.

size[source]

Current pool size.

Protocol Parser

The Reader class has two methods that are used when parsing replies from a stream of data. Reader.feed takes a string argument that is appended to the internal buffer. Reader.gets reads this buffer and returns a reply when the buffer contains a full reply. If a single call to feed contains multiple replies, gets should be called multiple times to extract all replies.

>>> reader = aiogibson.Reader()
>>> reader.feed(b'\x06\x00\x05\x03\x00\x00\x00bar')
>>> reader.gets()
b'bar'

When the buffer does not contain a full reply, gets returns False. This means extra data is needed and feed should be called again before calling gets again:

>>> reader.feed(b'')
>>> reader.gets()
False
>>> reader.feed(b'\x03\x00\x00\x00bar')
>>> reader.gets()
b'bar'
note:api same as in hiredis.

This module has encode_command, packs gibson command to binary format suitable to send over socket to gibson server:

>>> encode_command(b'set', 3600, 'foo', 3.14)
b'\x0f\x00\x00\x00\x01\x003600 foo 3.14'
aiogibson.parser.encode_command(command, *args)[source]

Pack and encode gibson command according to gibson binary protocol

See:

http://gibson-db.in/protocol/
Parameters:
  • command – bytes, gibson command (get, set, etc.)
  • args – required arguments for given command.
Returns:

bytes packed and encoded command.
class aiogibson.parser.Reader[source]

This class is responsible for parsing replies from the stream of data that is read from a Gibson connection. It does not contain functionality to handle I/O

feed(data)[source]

Put raw chunk of data obtained from connection to buffer.

Parameters:data – bytes, raw input data.
gets()[source]

When the buffer does not contain a full reply, gets returns False. This means extra data is needed and feed should be called again before calling gets again:

Returns:False there is no full reply or parsed obj.

Exceptions

exception aiogibson.errors.GibsonError[source]

Base exception class for aiogibson exceptions.

exception aiogibson.errors.ProtocolError[source]

Raised when protocol error occurs.

exception aiogibson.errors.ReplyError[source]

Generic error while executing the query

exception aiogibson.errors.ExpectedANumber[source]

Expected a number ( TTL or TIME ) but the specified value was invalid.

exception aiogibson.errors.MemoryLimitError[source]

The server reached configuration memory limit and will not accept any new value until its freeing routine will be executed.

exception aiogibson.errors.KeyLockedError[source]

The specified key was locked by a OP_LOCK or a OP_MLOCK query.

High Level Commands

aiogibson.commands.create_gibson(address, *, encoding=None, commands_factory=<class 'aiogibson.commands.Gibson'>, loop=None)[source]

Create high-level Gibson interface.

Parameters:
  • address – str for unix socket path, or tuple for (host, port) tcp connection.
  • encoding – this argument can be used to decode byte-replies to strings. By default no decoding is done.
  • commands_factory –
  • loop – event loop to use
Returns:

high-level Gibson connection Gibson
class aiogibson.commands.Gibson(connection)[source]

High-level Gibson interface

See:http://gibson-db.in/commands/
closed[source]

True if connection is closed.

count(prefix)[source]

Count items for a given prefix.

Parameters:prefix – bytes The key prefix to use as expression
Returns:int number of elements
dec(key)[source]

Decrement by one the given key.

Parameters:key – bytes, key to decrement.
Returns:int decremented value in case of success
delete(key)[source]

Delete the given key.

Parameters:key – bytes key to delete.
Returns:bool true in case of success.
end()[source]

Disconnects from the client from gibson instance.

get(key)[source]

Get the value for a given key.

Parameters:key – bytes key to get.
Returns:bytes if value exists else None
inc(key)[source]

Increment by one the given key.

Parameters:key – bytes, key to increment.
Returns:int incremented value
keys(prefix)[source]

Return a list of keys matching the given prefix.

Parameters:prefix – key prefix to use as expression.
Returns:list of available keys
lock(key, expire=0)[source]

Prevent the given key from being modified for a given amount of seconds.

Parameters:
  • key – bytes, key to decrement.
  • expire – int, time in seconds to lock the item.
Returns:

bool
Raises TypeError:
 

if expire argument is not int
mdec(prefix)[source]

Decrement by one keys verifying the given prefix.

Parameters:prefix – prefix for keys.
Returns:int, number of modified items, otherwise an error.
mdelete(prefix)[source]

Delete keys verifying the given prefix.

Parameters:prefix – prefix for keys.
Returns:int, number of modified items, otherwise an error.
meta_access(key)[source]

Timestamp of the last time the item was accessed.

Parameters:key – bytes, key of interest.
Returns:int, timestamp
meta_created(key)[source]

Timestamp of item creation.

Parameters:key – bytes, key of interest.
Returns:int, timestamp
meta_encoding(key)[source]

Gibson encoding for given value.

Parameters:key – bytes, key of interest.
Returns:int, gibson encoding, 0 - bytes, 2 - int.
meta_left(key)[source]

Number of seconds left for the item to live if a ttl was specified, otherwise -1.

Parameters:key – bytes, key of interest.
Returns:int, Number of seconds left.
meta_lock(key)[source]

Number of seconds the item is locked, -1 if there’s no lock.

Parameters:key – bytes, key of interest.
Returns:int, number of seconds
meta_size(key)[source]

The size in bytes of the item value.

Parameters:key – bytes, key of interest.
Returns:int, value size in bytes
meta_ttl(key)[source]

Item specified time to live, -1 for infinite TTL.

Parameters:key – bytes, key of interest.
Returns:int, seconds of TTL.
mget(prefix, limit=None)[source]

Get the values for keys with given prefix.

Parameters:
  • prefix – prefix for keys.
  • limit – maximum number of returned key/value paris.
Returns:

list of key/value pairs
Raises TypeError:
 

if limit argument is not int
minc(prefix)[source]

Increment by one keys verifying the given prefix.

Parameters:prefix – prefix for keys.
Returns:int, number of modified items, otherwise an error.
mlock(prefix, expire=0)[source]

Prevent keys verifying the given prefix from being modified for a given amount of seconds.

Parameters:prefix – bytes, prefix for keys.

:param expire:int, lock period in seconds. :return: int, number of modified items, otherwise an error. :raises TypeError: if expire argument is not int

mset(prefix, value)[source]

Set the value for keys verifying the given prefix.

Parameters:prefix – prefix for keys.
Returns:int, number of modified items, otherwise an error.
mttl(prefix, expire=0)[source]

Set the TTL for keys verifying the given prefix.

Parameters:
  • prefix – prefix for keys.
  • expire – int, new expiration time.
Returns:

int, number of modified items, otherwise an error.
Raises TypeError:
 

if expire argument is not int
munlock(prefix)[source]

Remove the lock on keys verifying the given prefix.

Parameters:prefix – prefix for keys.
Returns:int, number of affected items, otherwise an error.
ping()[source]

Ping the server instance to refresh client last seen timestamp.

Returns:True or error.
set(key, value, expire=0)[source]

Set the value for the given key, with an optional TTL.

Parameters:
  • key – bytes key to set.
  • value – bytes value to set.
  • expire – int optional ttl in seconds
Raises TypeError:
 

if expire argument is not int
stats()[source]

Get system stats about the Gibson instance.

Returns:list of pairs (stat, value).
ttl(key, expire)[source]

Set the TTL of a key.

Parameters:
  • key – bytes, key to set ttl.
  • expire – int, TTL in seconds.
Returns:

bool, True in case of success.
Raises TypeError:
 

if expire argument is not int
unlock(key)[source]

Remove the lock from the given key.

Parameters:key – bytes key ot unlock.
Returns:bool, True in case of success.

Indices and tables