barman.command_wrappers module#

This module contains a wrapper for shell commands

class barman.command_wrappers.BarmanSubProcess(command='/home/runner/work/barman-internal/barman-internal/src/.tox/docs/bin/sphinx-build', subcommand=None, config=None, args=None, keep_descriptors=False)View on GitHub #

Bases: object

Wrapper class for barman sub instances

__init__(command='/home/runner/work/barman-internal/barman-internal/src/.tox/docs/bin/sphinx-build', subcommand=None, config=None, args=None, keep_descriptors=False)View on GitHub #

Build a specific wrapper for all the barman sub-commands, providing a unified interface.

Parameters:

command (str) – path to barman
subcommand (str) – the barman sub-command
config (str) – path to the barman configuration file.
args (list[str]) – a list containing the sub-command args like the target server name
keep_descriptors (bool) – whether to keep the subprocess stdin, stdout, stderr descriptors attached. Defaults to False

execute()View on GitHub #: Execute the command and pass the output to the configured handlers

class barman.command_wrappers.Command(cmd, args=None, env_append=None, path=None, shell=False, check=False, allowed_retval=(0,), close_fds=True, out_handler=None, err_handler=None, retry_times=0, retry_sleep=0, retry_handler=None)View on GitHub #

Bases: object

Wrapper for a system command

__init__(cmd, args=None, env_append=None, path=None, shell=False, check=False, allowed_retval=(0,), close_fds=True, out_handler=None, err_handler=None, retry_times=0, retry_sleep=0, retry_handler=None)View on GitHub #

If the args argument is specified the arguments will be always added to the ones eventually passed with the actual invocation.

If the env_append argument is present its content will be appended to the environment of every invocation.

The subprocess output and error stream will be processed through the output and error handler, respectively defined through the out_handler and err_handler arguments. If not provided every line will be sent to the log respectively at INFO and WARNING level.

The out_handler and the err_handler functions will be invoked with one single argument, which is a string containing the line that is being processed.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

If retry_times is greater than 0, when the execution of a command terminates with an error, it will be retried for a maximum of retry_times times, waiting for retry_sleep seconds between every attempt.

Every time a command is retried the retry_handler is executed before running the command again. The retry_handler must be a callable that accepts the following fields:

the Command object

the arguments list

the keyword arguments dictionary

the number of the failed attempt

the exception containing the error

An example of such a function is:

> def retry_handler(command, args, kwargs, attempt, exc): > print(“Failed command!”)

Some of the keyword arguments can be specified both in the class constructor and during the method call. If specified in both places, the method arguments will take the precedence over the constructor arguments.

Parameters:

cmd (str) – The command to execute
args (list[str]|None) – List of additional arguments to append
env_append (dict[str.str]|None) – additional environment variables
path (str) – PATH to be used while searching for cmd
shell (bool) – If true, use the shell instead of an “execve” call
check (bool) – Raise a CommandFailedException if the exit code is not present in allowed_retval
allowed_retval (list[int]) – List of exit codes considered as a successful termination.
close_fds (bool) – If set, close all the extra file descriptors
out_handler (callable) – handler for lines sent on stdout
err_handler (callable) – handler for lines sent on stderr
retry_times (int) – number of allowed retry attempts
retry_sleep (int) – wait seconds between every retry
retry_handler (callable) – handler invoked during a command retry

_build_pipe(args, close_fds)View on GitHub #

Build the Pipe object used by the Command

The resulting command will be composed by:: self.cmd + self.args + args

Parameters:

args – extra arguments for the subprocess
close_fds – if True all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

Return type:

subprocess.Popen

_get_output_once(*args, **kwargs)View on GitHub #

Run the command and return the output and the error as a tuple.

The return code is not returned, but it can be accessed as an attribute of the Command object, as well as the output and the error strings.

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

Every keyword argument can be specified both in the class constructor and during the method call. If specified in both places, the method arguments will take the precedence over the constructor arguments.

Return type:: tuple[str, str]
Raises:: CommandFailedException

static _restore_sigpipe()View on GitHub #: restore default signal handler (http://bugs.python.org/issue1652)

check_return_value(allowed_retval)View on GitHub #

Check the current return code and raise CommandFailedException when it’s not in the allowed_retval list

Parameters:: allowed_retval (list[int]) – list of return values considered success
Raises:: CommandFailedException

enable_signal_forwarding(signal_id)View on GitHub #

Enable signal forwarding to the subprocess for a specified signal_id

Parameters:: signal_id – The signal id to be forwarded

execute(*args, **kwargs)View on GitHub #

Execute the command and pass the output to the configured handlers

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

Return type:: int
Raise:: CommandFailedException

get_output(*args, **kwargs)View on GitHub #

Run the command and return the output and the error as a tuple.

The return code is not returned, but it can be accessed as an attribute of the Command object, as well as the output and the error strings.

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

Return type:: tuple[str, str]
Raise:: CommandFailedException
Raise:: CommandMaxRetryExceeded

classmethod make_logging_handler(level, prefix=None)View on GitHub #

Build a handler function that logs every line it receives.

The resulting callable object logs its input at the specified level with an optional prefix.

Parameters:

level – The log level to use
prefix – An optional prefix to prepend to the line

Returns:

handler function

static make_output_handler(prefix=None)View on GitHub #

Build a handler function which prints every line it receives.

The resulting function prints (and log it at INFO level) its input with an optional prefix.

Parameters:: prefix – An optional prefix to prepend to the line
Returns:: handler function

static pipe_processor_loop(processors)View on GitHub #

Process the output received through the pipe until all the provided StreamLineProcessor reach the EOF.

Parameters:: processors (list[StreamLineProcessor]) – a list of StreamLineProcessor

class barman.command_wrappers.Handler(logger, level, prefix=None)View on GitHub #

Bases: object

__init__(logger, level, prefix=None)View on GitHub #

run(line)View on GitHub #

class barman.command_wrappers.PgBaseBackup(connection, destination, command, version=None, app_name=None, bwlimit=None, tbs_mapping=None, immediate=False, check=True, compression=None, parent_backup_manifest_path=None, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_basebackup system command

COMMAND_ALTERNATIVES = ['pg_basebackup']#

Sometimes the name of a command has been changed during the PostgreSQL evolution. I.e. that happened with pg_receivexlog, that has been renamed to pg_receivewal. In that case, we should try using pg_receivewal (the newer alternative) and, if that command doesn’t exist, we should try using pg_receivexlog.

This is a list of command names to be used to find the installed command.

__init__(connection, destination, command, version=None, app_name=None, bwlimit=None, tbs_mapping=None, immediate=False, check=True, compression=None, parent_backup_manifest_path=None, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
destination (str) – destination directory path
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
bwlimit (str) – bandwidth limit for pg_basebackup
tbs_mapping (Dict[str, str]) – used for tablespace
immediate (bool) – fast checkpoint identifier for pg_basebackup
check (bool) – check if the return value is in the list of allowed values of the Command obj
compression (barman.compression.PgBaseBackupCompression) – the pg_basebackup compression options used for this backup
parent_backup_manifest_path (str) – the path to a backup_manifest file from a previous backup which can be used to perform an incremental backup
args (List[str]) – additional arguments

_get_compression_args(version, compression)View on GitHub #

Determine compression related arguments for pg_basebackup from the supplied compression options in the format required by the pg_basebackup version.

Parameters:

version (Version) – The pg_basebackup version for which the arguments should be formatted.
compression (barman.compression.PgBaseBackupCompression) – the pg_basebackup compression options used for this backup

class barman.command_wrappers.PgCombineBackup(destination, command, tbs_mapping=None, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_combinebackup system command

COMMAND_ALTERNATIVES = ['pg_combinebackup']#

This is a list of command names to be used to find the installed command.

__init__(destination, command, tbs_mapping=None, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

destination (str) – destination directory path
command (str) – the command to use
tbs_mapping (None|Dict[str, str]) – used for tablespace
connection (PostgreSQL) – an object representing a database connection
version (Version) – the command version
app_name (str) – the application name to use for the connection
check (bool) – check if the return value is in the list of allowed values of the Command obj
args (None|List[str]) – additional arguments

class barman.command_wrappers.PgReceiveXlog(connection, destination, command, version=None, app_name=None, synchronous=False, check=True, slot_name=None, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for pg_receivexlog

COMMAND_ALTERNATIVES = ['pg_receivewal', 'pg_receivexlog']#

This is a list of command names to be used to find the installed command.

__init__(connection, destination, command, version=None, app_name=None, synchronous=False, check=True, slot_name=None, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
destination (str) – destination directory path
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
synchronous (bool) – request synchronous WAL streaming
check (bool) – check if the return value is in the list of allowed values of the Command obj
slot_name (str) – the replication slot name to use for the connection
args (List[str]) – additional arguments

class barman.command_wrappers.PgVerifyBackup(data_path, command, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_verify system command

COMMAND_ALTERNATIVES = ['pg_verifybackup']#

This is a list of command names to be used to find the installed command.

__init__(data_path, command, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Constructor :param str data_path: backup data directory :param str command: the command to use :param PostgreSQL connection: an object representing

a database connection

Parameters:

version (Version) – the command version
app_name (str) – the application name to use for the connection
check (bool) – check if the return value is in the list of allowed values of the Command obj
args (List[str]) – additional arguments

class barman.command_wrappers.PostgreSQLClient(connection, command, version=None, app_name=None, path=None, **kwargs)View on GitHub #

Bases: Command

Superclass of all the PostgreSQL client commands.

COMMAND_ALTERNATIVES = None#

This is a list of command names to be used to find the installed command.

__init__(connection, command, version=None, app_name=None, path=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
path (str) – additional path for executable retrieval

classmethod find_command(path=None)View on GitHub #

Find the active command, given all the alternatives as set in the property named COMMAND_ALTERNATIVES in this class.

Parameters:: path (str) – The path to use while searching for the command
Return type:: Command

classmethod get_version_info(path=None)View on GitHub #

Return a dictionary containing all the info about the version of the PostgreSQL client

Parameters:: path (str) – the PATH env

class barman.command_wrappers.Rsync(rsync='rsync', args=None, ssh=None, ssh_options=None, bwlimit=None, exclude=None, exclude_and_protect=None, include=None, network_compression=None, path=None, **kwargs)View on GitHub #

Bases: Command

This class is a wrapper for the rsync system command, which is used vastly by barman

__init__(rsync='rsync', args=None, ssh=None, ssh_options=None, bwlimit=None, exclude=None, exclude_and_protect=None, include=None, network_compression=None, path=None, **kwargs)View on GitHub #

Parameters:

rsync (str) – rsync executable name
args (list[str]|None) – List of additional argument to always append
ssh (str) – the ssh executable to be used when building the -e argument
ssh_options (list[str]) – the ssh options to be used when building the -e argument
bwlimit (str) – optional bandwidth limit
exclude (list[str]) – list of file to be excluded from the copy
exclude_and_protect (list[str]) – list of file to be excluded from the copy, preserving the destination if exists
include (list[str]) – list of files to be included in the copy even if excluded.
network_compression (bool) – enable the network compression
path (str) – PATH to be used while searching for cmd
check (bool) – Raise a CommandFailedException if the exit code is not present in allowed_retval
allowed_retval (list[int]) – List of exit codes considered as a successful termination.

_args_for_suse(args)View on GitHub #

Mangle args for SUSE compatibility

See https://bugzilla.opensuse.org/show_bug.cgi?id=898513

from_file_list(filelist, src, dst, *args, **kwargs)View on GitHub #

This method copies filelist from src to dst.

Returns the return code of the rsync command

get_output(*args, **kwargs)View on GitHub #: Run the command and return the output and the error (if present)

class barman.command_wrappers.RsyncPgData(rsync='rsync', args=None, **kwargs)View on GitHub #

Bases: Rsync

This class is a wrapper for rsync, specialised in sync-ing the Postgres data directory

__init__(rsync='rsync', args=None, **kwargs)View on GitHub #

Constructor

Parameters:: rsync (str) – command to run

class barman.command_wrappers.StreamLineProcessor(fobject, handler)View on GitHub #

Bases: object

Class deputed to reading lines from a file object, using a buffered read.

NOTE: This class never call os.read() twice in a row. And is designed to work with the select.select() method.

__init__(fobject, handler)View on GitHub #

Parameters:

fobject (file) – The file that is being read
handler (callable) – The function (taking only one unicode string argument) which will be called for every line

fileno()View on GitHub #

Method used by select.select() to get the underlying file descriptor.

Return type:: the underlying file descriptor

process()View on GitHub #

Read the ready data from the stream and for each line found invoke the handler.

Return bool:: True when End Of File has been reached

barman.command_wrappers.full_command_quote(command, args=None)View on GitHub #

Produce a command with quoted arguments

Parameters:

command (str) – the command to be executed
args (list[str]) – the command arguments

Return type:

str

barman.command_wrappers.shell_quote(arg)View on GitHub #

Quote a string argument to be safely included in a shell command line.

Parameters:: arg (str) – The script argument
Returns:: The argument quoted