Bonobo

Module

bonobo

Contains all the tools you need to get started with the framework, including (but not limited to) generic transformations, readers, writers, and tools for writing and executing graphs and jobs.

All objects in this module are considered very safe to use, and backward compatibility when moving up from one version to another is maximal.

Graphs

• bonobo.structs.graphs.Graph

Nodes

• bonobo.nodes.CsvReader

• bonobo.nodes.CsvWriter

• bonobo.nodes.FileReader

• bonobo.nodes.FileWriter

• bonobo.nodes.Filter

• bonobo.nodes.FixedWindow

• bonobo.nodes.Format()

• bonobo.nodes.JsonReader

• bonobo.nodes.JsonWriter

• bonobo.nodes.LdjsonReader

• bonobo.nodes.LdjsonWriter

• bonobo.nodes.Limit

• bonobo.nodes.MapFields()

• bonobo.nodes.OrderFields()

• bonobo.nodes.PickleReader

• bonobo.nodes.PickleWriter

• bonobo.nodes.PrettyPrinter

• bonobo.nodes.RateLimited

• bonobo.nodes.Rename()

• bonobo.nodes.SetFields()

• bonobo.nodes.Tee()

• bonobo.nodes.UnpackItems()

• bonobo.nodes.count()

• bonobo.nodes.identity()

• bonobo.nodes.noop()

Other top-level APIs

• bonobo.create_reader()

• bonobo.create_strategy()

• bonobo.create_writer()

• bonobo.get_argument_parser()

• bonobo.get_examples_path()

• bonobo.inspect()

• bonobo.open_examples_fs()

• bonobo.open_fs()

• bonobo.parse_args()

• bonobo.run()

create_reader

create_strategy

create_strategy(name=None)

Create a strategy, or just returns it if it’s already one.

Parameters

name –

Returns

Strategy

create_writer

get_argument_parser

get_argument_parser(parser=None)

Creates an argument parser with arguments to override the system environment.

Api

bonobo.get_argument_parser

Parameters

_parser –

Returns

get_examples_path

get_examples_path(*pathsegments)

inspect

inspect(graph, *, plugins=None, services=None, strategy=None, format)

open_examples_fs

open_examples_fs(*pathsegments)

open_fs

open_fs(fs_url=None, *args, **kwargs)

Wraps fs.opener.registry.Registry.open_fs, with default to local current working directory and expanding ~ in path.

Parameters

• fs_url (str) – A filesystem URL

• parse_result (ParseResult) – A parsed filesystem URL.

• writeable (bool) – True if the filesystem must be writeable.

• create (bool) – True if the filesystem should be created if it does not exist.

• cwd (str) – The current working directory (generally only relevant for OS filesystems).

• default_protocol (str) – The protocol to use if one is not supplied in the FS URL (defaults to "osfs").

Returns

fs.base.FS object

parse_args

parse_args(mixed=None)

Context manager to extract and apply environment related options from the provided argparser result.

A dictionnary with unknown options will be yielded, so the remaining options can be used by the caller.

Api

bonobo.patch_environ

Parameters

mixed – ArgumentParser instance, Namespace, or dict.

Returns

run

run(graph, *, plugins=None, services=None, strategy=None)

Main entry point of bonobo. It takes a graph and creates all the necessary plumbing around to execute it.

The only necessary argument is a Graph instance, containing the logic you actually want to execute.

By default, this graph will be executed using the “threadpool” strategy: each graph node will be wrapped in a thread, and executed in a loop until there is no more input to this node.

You can provide plugins factory objects in the plugins list, this function will add the necessary plugins for interactive console execution and jupyter notebook execution if it detects correctly that it runs in this context.

You’ll probably want to provide a services dictionary mapping service names to service instances.

Parameters

• graph (Graph) – The Graph to execute.

• strategy (str) – The bonobo.execution.strategies.base.Strategy to use.

• plugins (list) – The list of plugins to enhance execution.

• services (dict) – The implementations of services this graph will use.

Return bonobo.execution.graph.GraphExecutionContext