GearFunction

GearFunction objects are RedGrease’s representation of RedisGears Gear functions. There are a couple of different classes of GearFunctions to be aware of.

_images/GearFunctions_UML.png

UML Relationships between different GearFunction

These function objects can be dynamically constructed in either your application code or in separate script files. They are constructed using either the GearsBuilder (blue) or any of the Readers (green):

These are responsible for reading data from different sources into records, either in batch on-demand or when some event occurs.

GearFunctions are created by “applying” operations, such as Map, GroupBy, Aggregate etc, to either any of the Readers, a GearsBuilder, or some other GearFunction.

With the exception of the GearsBuilder, “applying” an operation on a GearFunction, doesn’t actually modify it, but instead creates a new function with that operation added.

All Gear functions therefore consists of a chain of zero or more operations, with some reader at the beginning.

Some of the Reader types have additional reader-specific operations, that can only be “applied” as as their first operation.

Gear functions are “terminated” by either one of two special Actions. Either Run, for immediate “batch-mode” execution, or Register, for registering the function for “event-mode” execution.

Some Readers support both “batch-mode” and “event-mode”, but some only support one of the modes.

GearFunctions that have not been “terminated” by either of the Actions, are referred to as an Open GearFunction, as they can be extended with more operations, creating new GearFunctions.

GearFunctions that have been “terminated” by either of the Actions, are referred to as a Closed GearFunction, as they cannot be extended with more operations, creating new GearFunctions, but they can be executed.

Note

This “open/closed” terminology is not explicitly used by RedisLabs in their RedisGears Function Documentation, but it was the most intuitive terminology I could think of, to describe how their design of GearsBuilder actually behaves.

Early versions of RedGrease used the term “partial” instead of “open”. This was deemed a bit confusing, because “partial functions” is a very specific, but completely different, thing in mathematics. It was also sometimes confused with “partial application”, which in computer science is yet another completely different (but powerful) concept.

Every Open GearFunction, including the GearsBuilder, implement the default set of operations.

When a GearFunction is executed, the Reader reads its data, and pass each record to its first operation, which modifies, filters or aggregates these records into some new output records, which in turn are passed to the next operation and so on, until the last operation.

The output of the final operation is then either, returned to the caller if it was a “batch-mode” execution and unblocking was not set to True in Gears.pyexecute(), or stored for later retrieval otherwise.

Open GearFunction

You would never instantiate a gears.OpenGearFunction yourself, but all “open” GearFunctions that has not yet been “closed” with the Run or Register Actions, inherits from this class.

It is this class that under the hood is responsible for “applying” operations and Actions, and thus creating new GearFunctions.

This includes both the runtime.GearsBuilder as well as the Readers:

class redgrease.gears.OpenGearFunction(operation: redgrease.gears.Operation, input_function: Optional[redgrease.gears.OpenGearFunction] = None, requirements: Optional[Iterable[str]] = None)[source]

An open Gear function is a Gear function that is not yet “closed” with a Run action or a Register action.

Open Gear functions can be used to create new “open” gear functions by applying operations, or it can create a closed Gear function by applying either the Run action or a Register action.

run(arg: Optional[str] = None, convertToStr: bool = True, collect: bool = True, requirements: Optional[Iterable[str]] = None, on=None, **kwargs) redgrease.gears.ClosedGearFunction[redgrease.typing.InputRecord][source]

Create a “closed” function to be Run as in “batch-mode”.

Batch functions are executed once and exits once the data is exhausted by its reader.

Parameters
  • arg (str, optional) –

    An optional argument that’s passed to the reader as its defaultArg. It means the following:

    • A glob-like pattern for the KeysReader and KeysOnlyReader readers.

    • A key name for the StreamReader reader.

    • A Python generator for the PythonReader reader.

    Defaults to None.

  • convertToStr (bool, optional) – When True, adds a map operation to the flow’s end that stringifies records. Defaults to False.

  • collect (bool, optional) – When True adds a collect operation to flow’s end. Defaults to False.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • on (redis.Redis) – Immediately execute the function on this RedisGears system.

  • **kwargs – Additional parameters to the run operation.

Returns

A new closed batch function, if on is not specified. An execution result, if on is specified.

Return type

Union[ClosedGearFunction, redgrease.data.ExecutionResult]

Raises

TypeError – If the function does not support batch mode.

register(prefix: str = '*', convertToStr: bool = True, collect: bool = True, mode: Optional[str] = None, onRegistered: Optional[Callable[[], None]] = None, eventTypes: Optional[Iterable[str]] = None, keyTypes: Optional[Iterable[str]] = None, readValue: Optional[bool] = None, batch: Optional[int] = None, duration: Optional[int] = None, onFailedPolicy: Optional[str] = None, onFailedRetryInterval: Optional[int] = None, trimStream: Optional[bool] = None, trigger: Optional[str] = None, requirements: Optional[Iterable[str]] = None, on=None, **kwargs) redgrease.gears.ClosedGearFunction[redgrease.typing.InputRecord][source]

Create a “closed” function to be Register ‘ed as an event-triggered function.

Event functions are executed each time an event arrives. Each time it is executed, the function operates on the event’s data and once done is suspended until its future invocations by new events. :param prefix: Key prefix pattern to match on.

Not relevant for ‘CommandReader’ readers (see ‘trigger’). Defaults to "*".

Parameters
  • convertToStr (bool, optional) – When True adds a map operation to the flow’s end that stringifies records. Defaults to True.

  • collect (bool, optional) – When True adds a collect operation to flow’s end. Defaults to False.

  • mode (str, optional) –

    The execution mode of the function. Can be one of:

    • "async":

      Execution will be asynchronous across the entire cluster.

    • "async_local":

      Execution will be asynchronous and restricted to the handling shard.

    • "sync":

      Execution will be synchronous and local.

    Defaults to "async".

  • onRegistered (Registrator, optional) – A function that’s called on each shard upon function registration. It is a good place to initialize non-serializeable objects such as network connections. Defaults to None.

  • eventTypes (Iterable[str], optional) –

    For KeysReader only. A whitelist of event types that trigger execution when the KeysReader are used. The list may contain one or more:

    • Any Redis or module command

    • Any Redis event

    Defaults to None.

  • keyTypes (Iterable[str], optional) –

    For KeysReader and KeysOnlyReader only. A whitelist of key types that trigger execution when using the KeysReader or KeysOnlyReader readers. The list may contain one or more from the following:

    • Redis core types:

      "string", "hash", "list", "set", "zset" or "stream"

    • Redis module types:

      "module"

    Defaults to None.

  • readValue (bool, optional) – For KeysReader only. When False the value will not be read, so the ‘type’ and ‘value’ of the record will be set to None. Defaults to True.

  • batch (int, optional) – For StreamReader only. The number of new messages that trigger execution. Defaults to 1.

  • duration (int, optional) – For StreamReader only. The time to wait before execution is triggered, regardless of the batch size (0 for no duration). Defaults to 0.

  • onFailedPolicy (str, optional) –

    For StreamReader only. The policy for handling execution failures. May be one of:

    • "continue":

      Ignores a failure and continues to the next execution. This is the default policy.

    • "abort":

      Stops further executions.

    • "retry":

      Retries the execution after an interval specified with onFailedRetryInterval (default is one second).

    Defaults to "continue".

  • onFailedRetryInterval (int, optional) – For StreamReader only. The interval (in milliseconds) in which to retry in case onFailedPolicy is ‘retry’. Defaults to 1.

  • trimStream (bool, optional) – For StreamReader only. When True the stream will be trimmed after execution Defaults to True.

  • trigger (str) – For ‘CommandReader’ only, and mandatory. The trigger string that will trigger the function.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • on (redis.Redis) – Immediately execute the function on this RedisGears system.

  • **kwargs – Additional parameters to the register operation.

Returns

A new closed event function, if on is not specified. An execution result, if on is specified.

Return type

Union[ClosedGearFunction, redgrease.data.ExecutionResult]

Raises

TypeError – If the function does not support event mode.

map()[source]

Instance-local Map operation that performs a one-to-one (1:1) mapping of records.

Parameters
  • op (redgrease.typing.Mapper) – Function to map on the input records. The function must take one argument as input (input record) and return something as an output (output record).

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Map operation.

Returns

A new “open” gear function with a Map operation as last step.

Return type

OpenGearFunction

flatmap()[source]

Instance-local FlatMap operation that performs one-to-many (1:N) mapping of records.

Parameters
  • op (redgrease.typing.Expander, optional) – Function to map on the input records. The function must take one argument as input (input record) and return an iterable as an output (output records). Defaults to the ‘identity-function’, I.e. if input is an iterable will be expanded.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the FlatMap operation.

Returns

A new “open” gear function with a FlatMap operation as last step.

Return type

OpenGearFunction

foreach()[source]

Instance-local ForEach operation performs one-to-the-same (1=1) mapping.

Parameters
  • op (redgrease.typing.Processor) – Function to run on each of the input records. The function must take one argument as input (input record) and should not return anything.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the ForEach operation.

Returns

A new “open” gear function with a ForEach operation as last step.

Return type

OpenGearFunction

filter()[source]

Instance-local Filter operation performs one-to-zero-or-one (1:bool) filtering of records.

Parameters
  • op (redgrease.typing.Filterer, optional) – Function to apply on the input records, to decide which ones to keep. The function must take one argument as input (input record) and return a bool. The input records evaluated to True will be kept as output records. Defaults to the ‘identity-function’, i.e. records are filtered based on their own trueness or falseness.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Filter operation.

Returns

A new “open” gear function with a Filter operation as last step.

Return type

OpenGearFunction

accumulate(op: Optional[Callable[[redgrease.gears.T, redgrease.typing.InputRecord], redgrease.gears.T]] = None, requirements: Optional[Iterable[str]] = None, **kwargs) redgrease.gears.OpenGearFunction[redgrease.gears.T][source]

Instance-local Accumulate operation performs many-to-one mapping (N:1) of records.

Parameters
  • op (redgrease.typing.Accumulator, optional) –

    Function to to apply on the input records. The function must take two arguments as input:

    • An accumulator value, and

    • The input record.

    It should aggregate the input record into the accumulator variable, which stores the state between the function’s invocations. The function must return the accumulator’s updated value. Defaults to a list accumulator, I.e. the output will be a list of all inputs.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Accumulate operation.

Returns

A new “open” gear function with Accumulate operation as last step.

Return type

OpenGearFunction

localgroupby()[source]

Instance-local LocalGroupBy operation performs many-to-less mapping (N:M) of records.

Parameters
  • extractor (redgrease.typing.Extractor, optional) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to the hash of the input.

  • reducer (redgrease.typing.Reducer, optional) – Function to apply on the records of each group, to reduce to a single value (per group). The function must take (a) a key, (b) an input record and (c) a variable that’s called an accumulator. It performs similarly to the accumulator callback, with the difference being that it maintains an accumulator per reduced key / group. Defaults to a list accumulator, I.e. the output will be a list of all inputs, for each group.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the LocalGroupBy operation.

Returns

A new “open” gear function with a LocalGroupBy operation as last step.

Return type

OpenGearFunction

limit()[source]

Instance-local Limit operation limits the number of records.

Parameters
  • length (int) – The maximum number of records.

  • start (int, optional) – The index of the first input record. Defaults to 0.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Limit operation.

Returns

A new “open” gear function with a Limit operation as last step.

Return type

OpenGearFunction

collect()[source]

Cluster-global Collect operation collects the result records.

Parameters

**kwargs – Additional parameters to the Collect operation.

Returns

A new “open” gear function with a Collect operation as last step.

Return type

OpenGearFunction

repartition()[source]

Cluster-global Repartition operation repartitions the records by shuffling them between shards.

Parameters
  • extractor (Extractor) – Function that takes a record and calculates a key that is used to determine the hash slot, and consequently the shard, that the record should migrate to to. The function must take one argument as input (input record) and return a string (key). The hash slot, and consequently the destination shard, is determined by the value of the key.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Repartition operation.

Returns

A new “open” gear function with a Repartition operation as last step.

Return type

OpenGearFunction

aggregate(zero: Optional[redgrease.gears.T] = None, seqOp: Optional[Callable[[redgrease.gears.T, redgrease.typing.InputRecord], redgrease.gears.T]] = None, combOp: Optional[Callable[[redgrease.gears.T, redgrease.gears.T], redgrease.gears.T]] = None, requirements: Optional[Iterable[str]] = None, **kwargs) redgrease.gears.OpenGearFunction[redgrease.gears.T][source]

Distributed Aggregate operation perform an aggregation on local data then a global aggregation on the local aggregations.

Parameters
  • zero (Any, optional) – The initial / zero value of the accumulator variable. Defaults to an empty list.

  • seqOp (redgrease.typing.Accumulator, optional) – A function to be applied on each of the input records, locally per shard. It must take two parameters: - an accumulator value, from previous calls - an input record The function aggregates the input into the accumulator variable, which stores the state between the function’s invocations. The function must return the accumulator’s updated value. Defaults to addition, if ‘zero’ is a number and to a list accumulator if ‘zero’ is a list.

  • combOp (redgrease.typing.Accumulator, optional) – A function to be applied on each of the aggregated results of the local aggregation (i.e. the output of seqOp). It must take two parameters: - an accumulator value, from previous calls - an input record The function aggregates the input into the accumulator variable, which stores the state between the function’s invocations. The function must return the accumulator’s updated value. Defaults to re-use the seqOp function.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the ref:op_aggregate operation.

Returns

A new “open” gear function with a ref:op_aggregate operation as last step.

Return type

OpenGearFunction

aggregateby()[source]

Distributed AggregateBy operation, behaves like aggregate, but separated on each key, extracted using the extractor.

Parameters
  • extractor (redgrease.typing.Extractor, optional) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to the hash of the input.

  • zero (Any, optional) – The initial / zero value of the accumulator variable. Defaults to an empty list.

  • seqOp (redgrease.typing.Accumulator, optional) – A function to be applied on each of the input records, locally per shard and group. It must take two parameters: - an accumulator value, from previous calls - an input record The function aggregates the input into the accumulator variable, which stores the state between the function’s invocations. The function must return the accumulator’s updated value. Defaults to a list reducer.

  • combOp (redgrease.typing.Accumulator) – A function to be applied on each of the aggregated results of the local aggregation (i.e. the output of seqOp). It must take two parameters: - an accumulator value, from previous calls - an input record The function aggregates the input into the accumulator variable, which stores the state between the function’s invocations. The function must return the accumulator’s updated value. Defaults to re-use the seqOp function.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the AggregateBy operation.

Returns

A new “open” gear function with a AggregateBy operation as last step.

Return type

OpenGearFunction

groupby()[source]

Cluster-local GroupBy operation performing a many-to-less (N:M) grouping of records.

Parameters
  • extractor (redgrease.typing.Extractor, optional) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to the hash of the input.

  • reducer (redgrease.typing.Reducer, optional) – Function to apply on the records of each group, to reduce to a single value (per group). The function must take (a) a key, (b) an input record and (c) a variable that’s called an accumulator. It performs similarly to the accumulator callback, with the difference being that it maintains an accumulator per reduced key / group. Defaults to a list reducer.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the GroupBy operation.

Returns

A new “open” gear function with a GroupBy operation as last step.

Return type

OpenGearFunction

batchgroupby()[source]

Cluster-local GroupBy operation, performing a many-to-less (N:M) grouping of records.

Note: Using this operation may cause a substantial increase in memory usage

during runtime. Consider using the GroupBy

Parameters
  • extractor (redgrease.typing.Extractor, optional) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to the hash of the input.

  • reducer (redgrease.typing.Reducer) – Function to apply on the records of each group, to reduce to a single value (per group). The function must take (a) a key, (b) an input record and (c) a variable that’s called an accumulator. It performs similarly to the accumulator callback, with the difference being that it maintains an accumulator per reduced key / group. Default is the length (len) of the input.

  • **kwargs – Additional parameters to the GroupBy operation.

Returns

A new “open” gear function with a GroupBy operation as last step.

Return type

OpenGearFunction

sort()[source]

Sort the records

Parameters
  • reverse (bool, optional) – Sort in descending order (higher to lower). Defaults to True.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Sort operation.

Returns

A new “open” gear function with a Sort operation as last step.

Return type

OpenGearFunction

distinct()[source]

Keep only the Distinct values in the data.

Parameters

**kwargs – Additional parameters to the Distinct operation.

Returns

A new “open” gear function with a Distinct operation as last step.

Return type

OpenGearFunction

count(**kwargs) redgrease.gears.OpenGearFunction[int][source]

Count the number of records in the execution.

Parameters

**kwargs – Additional parameters to the Count operation.

Returns

A new “open” gear function with a Count operation as last step.

Return type

OpenGearFunction

countby(extractor: Callable[[redgrease.typing.InputRecord], Hashable] = <function OpenGearFunction.<lambda>>, requirements: Optional[Iterable[str]] = None, **kwargs) redgrease.gears.OpenGearFunction[Dict[Hashable, int]][source]

Distributed CountBy operation counting the records grouped by key.

Parameters
  • extractor (redgrease.typing.Extractor) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to lambda x: str(x).

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the CountBy operation.

Returns

A new “open” gear function with a CountBy operation as last step.

Return type

OpenGearFunction

avg(extractor: Callable[[redgrease.typing.InputRecord], float] = <function OpenGearFunction.<lambda>>, requirements: Optional[Iterable[str]] = None, **kwargs) redgrease.gears.OpenGearFunction[float][source]

Distributed Avg operation, calculating arithmetic average of the records.

Parameters
  • extractor (redgrease.typing.Extractor) – Function to apply on the input records, to extact the grouping key. The function must take one argument as input (input record) and return a string (key). The groups are defined by the value of the key. Defaults to lambda x: float(x).

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

  • **kwargs – Additional parameters to the Avg operation.

Returns

A new “open” gear function with an Avg operation as last step.

Return type

OpenGearFunction

Closed GearFunction

class redgrease.gears.ClosedGearFunction[source]

Closed Gear functions are GearsFunctions that have been “closed” with a Run action or a Register action.

Closed Gear functions cannot add more operations, but can be executed in RedisGears.

on(gears_server, unblocking: bool = False, requirements: Optional[Iterable[str]] = None, replace: Optional[bool] = None, **kwargs)[source]

Execute the function on a RedisGears. This is equivalent to passing the function to Gears.pyexecute

Parameters
  • gears_server ([type]) – Redis client / connection object.

  • unblocking (bool, optional) – Execute function unblocking, i.e. asynchronous. Defaults to False.

  • requirements (Iterable[str], optional) – Additional requirements / dependency Python packages. Defaults to None.

Returns

The result of the function, just as Gears.pyexecute

Return type

redgrease.data.ExecutionResult

GearsBuilder

If you are familiar with RedisGears from before, then the runtime.Gearsbuilder should be very familiar. In fact the RedGrease version is designed to be backwards compatible with the Context Builder of RedisGears, with the same name.

The the GearsBuilder is technically a part of on the Builtin Runtime Functions and is exposed both through redgrease.runtime.GearsBuilder as well as redgrease.GearsBuilder.

Check out the Builtin Runtime Functions and specifically the section on the GearsBuilder for more details.

Readers

KeysReader

class redgrease.reader.KeysReader(default_key_pattern: str = '*', desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

KeysReader is a convenience class for GearsBuilder(“KeysReader”, …)

Instantiate a KeysReader “open” Gear function.

Parameters
  • default_key_pattern (str, optional) – Default Redis key pattern for the keys (and its values, type) to read. Defaults to “*”.

  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

values(type=Ellipsis, event=Ellipsis) redgrease.gears.OpenGearFunction[source]

Filter out and select the values of the records only.

Parameters
  • type (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis type(s) of keys to select. Valid values include: “string”, “hash”, “list”, “set”, “zset”, “stream” or “module”. Defaults to … (Ellipsis), meaning any type.

  • event (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis command or event `that . Defaults to … (Ellipsis), meaning any event.

Returns

A new “open” gear function generating the matching values.

Return type

OpenGearFunction

keys(type=Ellipsis, event=Ellipsis) redgrease.gears.OpenGearFunction[str][source]

Filter out and select the keys of the records only.

Parameters
  • type (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis type(s) of keys to select. Valid values include: “string”, “hash”, “list”, “set”, “zset”, “stream” or “module”. Defaults to … (Ellipsis), meaning any type.

  • event (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis command or event `that . Defaults to … (Ellipsis), meaning any event.

Returns

A new “open” gear function generating the matching keys.

Return type

OpenGearFunction[str]

records(type=Ellipsis, event=Ellipsis) redgrease.gears.OpenGearFunction[redgrease.utils.Record][source]

Filter out and map the records to redgrease.utils.Record objects.

This provides the fields key, value, type and event as typed attributes on an object, instead of items in a dict, making it a little bit more pleasant to work with.

Parameters
  • type (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis type(s) of keys to select. Valid values include: “string”, “hash”, “list”, “set”, “zset”, “stream” or “module”. Defaults to … (Ellipsis), meaning any type.

  • event (Union[str, Container[str]], optional) – A single string, or a container of several strings, representing the Redis command or event `that. Defaults to … (Ellipsis), meaning any event.

Returns

A new “open” gear function generating the matching Record values.

Return type

OpenGearFunction[redgrease.utils.Record]

KeysOnlyReader

class redgrease.reader.KeysOnlyReader(default_key_pattern: str = '*', desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

KeysOnlyReader is a convenience class for GearsBuilder(“KeysOnlyReader”, …)

Instantiate a KeysOnlyReader “open” Gear function.

Parameters
  • default_key_pattern (str, optional) – Default Redis keys pattern for the keys to read. Defaults to “*”.

  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

StreamReader

class redgrease.reader.StreamReader(default_key_pattern: str = '*', desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

StreamReader is a convenience class for GearsBuilder(“StreamReader”, …)

Instantiate a StreamReader “open” Gear function.

Parameters
  • default_key_pattern (str, optional) – Default Redis keys pattern for the redis stream(s) to read. Defaults to “*”.

  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

values() redgrease.gears.OpenGearFunction[Dict][source]

Select the values of the stream only.

Returns

A new “open” gear function generating values.

Return type

OpenGearFunction

keys() redgrease.gears.OpenGearFunction[str][source]

Select the keys, i.e. stream names, only.

Returns

A new “open” gear function generating names.

Return type

OpenGearFunction[str]

records() redgrease.gears.OpenGearFunction[redgrease.utils.StreamRecord][source]

Filter out and map the records to redgrease.utils.StreamRecord objects.

This provides the fields key, id and value as typed attributes on an object, instead of items in a dict, making it a little bit more pleasant to work with.

Returns

A new “open” gear function generating the matching Record values.

Return type

OpenGearFunction[redgrease.utils.Record]

PythonReader

class redgrease.reader.PythonReader(desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

PythonReader is a convenience class for GearsBuilder(“PythonReader”, …)

Instantiate a PythonReader “open” Gear function.

Parameters
  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

ShardsIDReader

class redgrease.reader.ShardsIDReader(desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

ShardsIDReader is a convenience class for GearsBuilder(“ShardsIDReader”, …)

Instantiate a ShardsIDReader “open” Gear function.

Parameters
  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

CommandReader

class redgrease.reader.CommandReader(desc: Optional[str] = None, requirements: Optional[Iterable[str]] = None)[source]

CommandReader is a convenience class for GearsBuilder(“CommandReader”, …)

Instantiate a CommandReader “open” Gear function.

Parameters
  • desc (str, optional) – An optional description. Defaults to None.

  • requirements (Iterable[str], optional) – Package dependencies for the gear train. Defaults to None.

args(max_count: Optional[int] = None) redgrease.gears.OpenGearFunction[source]

Ignore the trigger name and take only the arguments following the trigger.

Parameters

max_count (int, optional) – Maximum number of args to take. Any additional args will be truncated. Defaults to None.

Returns

A new “open” gear function only generating the trigger arguments.

Return type

redgrease.gears.OpenGearFunction

apply(fun: Callable[[...], redgrease.typing.OutputRecord], requirements: Optional[Iterable[str]] = None, **kwargs) redgrease.gears.OpenGearFunction[redgrease.typing.OutputRecord][source]

Apply a function to the trigger arguments.

Parameters

fun (Callable[…, redgrease.typing.OutputRecord]) – The function to call with the trigger arguments.

Returns

A new “open” gear function generating the results of the function.

Return type

redgrease.gears.OpenGearFunction[redgrease.typing.OutputRecord]


Courtesy of : Lyngon Pte. Ltd.