barman.command_wrappers module#

This module contains a wrapper for shell commands

class barman.command_wrappers.BarmanSubProcess(command='/home/runner/work/barman-internal/barman-internal/src/.tox/docs/bin/sphinx-build', subcommand=None, config=None, args=None, keep_descriptors=False)View on GitHub #

Bases: object

Wrapper class for barman sub instances

__init__(command='/home/runner/work/barman-internal/barman-internal/src/.tox/docs/bin/sphinx-build', subcommand=None, config=None, args=None, keep_descriptors=False)View on GitHub #

Build a specific wrapper for all the barman sub-commands, providing a unified interface.

Parameters:

command (str) – path to barman
subcommand (str) – the barman sub-command
config (str) – path to the barman configuration file.
args (list[str]) – a list containing the sub-command args like the target server name
keep_descriptors (bool) – whether to keep the subprocess stdin, stdout, stderr descriptors attached. Defaults to False

execute()View on GitHub #: Execute the command and pass the output to the configured handlers

class barman.command_wrappers.Command(cmd, args=None, env_append=None, path=None, skip_path_check=False, shell=False, check=False, allowed_retval=(0,), close_fds=True, wait=True, out_handler=None, err_handler=None, retry_times=0, retry_sleep=0, retry_handler=None)View on GitHub #

Bases: object

Wrapper for a system command

__init__(cmd, args=None, env_append=None, path=None, skip_path_check=False, shell=False, check=False, allowed_retval=(0,), close_fds=True, wait=True, out_handler=None, err_handler=None, retry_times=0, retry_sleep=0, retry_handler=None)View on GitHub #

If the args argument is specified the arguments will be always added to the ones eventually passed with the actual invocation.

If the env_append argument is present its content will be appended to the environment of every invocation.

If skip_path_check is True the command existence will not be checked during the object construction. This is useful if the path is checked earlier/later or in different ways e.g. on a remote host instead of locally.

The subprocess output and error stream will be processed through the output and error handler, respectively defined through the out_handler and err_handler arguments. If not provided every line will be sent to the log respectively at INFO and WARNING level.

The out_handler and the err_handler functions will be invoked with one single argument, which is a string containing the line that is being processed.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If wait is True this class only returns when the command has been terminated. Otherwise, it returns immediately after spawning the subprocess. When wait is False, the output/error/retry handlers are ignored. Similarly, retries are also not managed in case of failures that happen during the execution of the command.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

If retry_times is greater than 0, when the execution of a command terminates with an error, it will be retried for a maximum of retry_times times, waiting for retry_sleep seconds between every attempt.

Every time a command is retried the retry_handler is executed before running the command again. The retry_handler must be a callable that accepts the following fields:

the Command object

the arguments list

the keyword arguments dictionary

the number of the failed attempt

the exception containing the error

An example of such a function is:

> def retry_handler(command, args, kwargs, attempt, exc): > print(“Failed command!”)

Some of the keyword arguments can be specified both in the class constructor and during the method call. If specified in both places, the method arguments will take the precedence over the constructor arguments.

Parameters:

cmd (str) – The command to execute
args (list[str]|None) – List of additional arguments to append
env_append (dict[str.str]|None) – additional environment variables
path (str) – PATH to be used while searching for cmd
shell (bool) – If true, use the shell instead of an “execve” call
check (bool) – Raise a CommandFailedException if the exit code is not present in allowed_retval
allowed_retval (list[int]) – List of exit codes considered as a successful termination.
close_fds (bool) – If set, close all the extra file descriptors
wait (bool) – If True, it returns only when the command is terminated. Otherwise, it returns immediately after spawning the subprocess.
out_handler (callable) – handler for lines sent on stdout
err_handler (callable) – handler for lines sent on stderr
retry_times (int) – number of allowed retry attempts
retry_sleep (int) – wait seconds between every retry
retry_handler (callable) – handler invoked during a command retry

Note

The methods get_returncode() and get_stderr() are meant to be used when wait is set to False. For wait set to True, the return code and the stderr are available as attributes in the object, namely ret and err respectively.

_build_pipe(args, close_fds)View on GitHub #

Build the Pipe object used by the Command

The resulting command will be composed by:: self.cmd + self.args + args

Parameters:

args – extra arguments for the subprocess
close_fds – if True all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

Return type:

subprocess.Popen

_get_result(*args, **kwargs)View on GitHub #

Run the command and return the output and the error as a tuple.

The return code is not returned, but it can be accessed as an attribute of the Command object, as well as the output and the error strings.

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

If wait is True this method only returns when the command has been terminated. Otherwise, it returns immediately after spawning the subprocess, in which case (None, None) is returned.

Every keyword argument can be specified both in the class constructor and during the method call. If specified in both places, the method arguments will take the precedence over the constructor arguments.

Return type:: tuple[str, str]
Raises:: CommandFailedException

check_return_value(allowed_retval)View on GitHub #

Check the current return code and raise CommandFailedException when it’s not in the allowed_retval list

Parameters:: allowed_retval (list[int]) – list of return values considered success
Raises:: CommandFailedException

enable_signal_forwarding(signal_id)View on GitHub #

Enable signal forwarding to the subprocess for a specified signal_id

Parameters:: signal_id – The signal id to be forwarded

execute(*args, **kwargs)View on GitHub #

Execute the command and pass the output to the configured handlers

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

Return type:: int
Raise:: CommandFailedException

get_output(*args, **kwargs)View on GitHub #

Run the command and return the output and the error as a tuple.

The return code is not returned, but it can be accessed as an attribute of the Command object, as well as the output and the error strings.

If stdin argument is specified, its content will be passed to the executed command through the standard input descriptor.

If the close_fds argument is True, all file descriptors except 0, 1 and 2 will be closed before the child process is executed.

If the check argument is True, the exit code will be checked against the allowed_retval list, raising a CommandFailedException if not in the list.

If wait is True this method only returns when the command has been terminated. Otherwise, it returns immediately after spawning the subprocess.

Return type:: tuple[str, str]
Raise:: CommandFailedException
Raise:: CommandMaxRetryExceeded

get_returncode()View on GitHub #

Get the return code of the command if it has terminated

Return type:: int|None
Returns:: the exit code of the command if it has terminated, None otherwise

get_stderr()View on GitHub #

Get the stderr of the command if it has terminated

Return type:: str|None
Returns:: the stderr of the command if it has terminated, None otherwise

is_running()View on GitHub #

Check if the command is still running

Return type:: bool
Returns:: True if the command is still running, False otherwise

classmethod make_logging_handler(level, prefix=None)View on GitHub #

Build a handler function that logs every line it receives.

The resulting callable object logs its input at the specified level with an optional prefix.

Parameters:

level – The log level to use
prefix – An optional prefix to prepend to the line

Returns:

handler function

static make_output_handler(prefix=None)View on GitHub #

Build a handler function which prints every line it receives.

The resulting function prints (and log it at INFO level) its input with an optional prefix.

Parameters:: prefix – An optional prefix to prepend to the line
Returns:: handler function

pause()View on GitHub #: Pause the command execution sending a SIGSTOP to the subprocess

property pidView on GitHub #

Get the pid of the subprocess

Return type:: int|None

static pipe_processor_loop(processors)View on GitHub #

Process the output received through the pipe until all the provided StreamLineProcessor reach the EOF.

Parameters:: processors (list[StreamLineProcessor]) – a list of StreamLineProcessor

resume()View on GitHub #: Resume the command execution sending a SIGCONT to the subprocess

terminate()View on GitHub #: Terminate the command sending a SIGTERM to the subprocess

wait_exit()View on GitHub #

Wait for the command to exit.

Return type:: int|None
Returns:: the exit code of the command

class barman.command_wrappers.GPG(gpg='gpg', action=None, recipient=None, input_filepath=None, output_filepath=None, **kwargs)View on GitHub #

Bases: Command

Wrapper class for the Gnu Privacy Guard (GPG) command-line tool.

This class provides an interface to encrypt and decrypt data using GPG.

Note

GPG is a complete and free implementation of the OpenPGP standard as defined by RFC4880.

It supports encryption, signing, and key management, and is designed for easy integration with other applications.

In this class, we are only interested in encryption/decryption.

Important

–pinentry-mode loopback requires GPG version >= 2.1.

Important

This class expects sensitive data such as a GPG passphrase to be passed as a bytearray, not as a str. Python strings are immutable and may linger in memory, potentially exposing sensitive data. Using a bytearray allows the caller to explicitly zero out the passphrase after use, similar to memset in C.

Example:: passphrase = bytearray(b”your-secret-passphrase”) gpg(stdin=passphrase) passphrase[:] = b”" * len(passphrase)

Example for decryption:

>>> gpg = GPG(
...     action="decrypt",
...     input_filepath="file.gpg",
...     output_filepath="file.decrypted"
... )
>>> passphrase = bytearray(b"secret")
>>> gpg(stdin=passphrase)
>>> passphrase[:] = b"" * len(passphrase)

Example for decryption:

>>> gpg = GPG(
...     action="encrypt",
...     input_filepath="file.txt",
...     output_filepath="file.txt.gpg",
...     recipient="user@example.com"
... )
>>> gpg()

__init__(gpg='gpg', action=None, recipient=None, input_filepath=None, output_filepath=None, **kwargs)View on GitHub #

Initialize the GPG command wrapper.

Parameters:

gpg (str) – Path or name of the GPG executable. Defaults to gpg.
action (str) – Action to perform: encrypt or decrypt.
recipient (str) – Key identifier for encryption (required if action is encrypt).
input_filepath (str) – File to encrypt or decrypt.
output_filepath (str) – Output file path for encrypted/decrypted data.
kwargs – Additional keyword arguments passed to the base class:Command class.

Raises:

CommandException – If encrypt is specified without a recipient.
ValueError – If action is invalid.

class barman.command_wrappers.Handler(logger, level, prefix=None)View on GitHub #

Bases: object

__init__(logger, level, prefix=None)View on GitHub #

run(line)View on GitHub #

class barman.command_wrappers.PgBaseBackup(connection, destination, command, version=None, app_name=None, bwlimit=None, tbs_mapping=None, immediate=False, check=True, compression=None, parent_backup_manifest_path=None, no_sync=False, warehousepg_dbid=None, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_basebackup system command

COMMAND_ALTERNATIVES = ['pg_basebackup']#

Sometimes the name of a command has been changed during the PostgreSQL evolution. I.e. that happened with pg_receivexlog, that has been renamed to pg_receivewal. In that case, we should try using pg_receivewal (the newer alternative) and, if that command doesn’t exist, we should try using pg_receivexlog.

This is a list of command names to be used to find the installed command.

__init__(connection, destination, command, version=None, app_name=None, bwlimit=None, tbs_mapping=None, immediate=False, check=True, compression=None, parent_backup_manifest_path=None, no_sync=False, warehousepg_dbid=None, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
destination (str) – destination directory path
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
bwlimit (str) – bandwidth limit for pg_basebackup
tbs_mapping (Dict[str, str]) – used for tablespace
immediate (bool) – fast checkpoint identifier for pg_basebackup
check (bool) – check if the return value is in the list of allowed values of the Command obj
compression (barman.compression.PgBaseBackupCompression) – the pg_basebackup compression options used for this backup
parent_backup_manifest_path (str) – the path to a backup_manifest file from a previous backup which can be used to perform an incremental backup
no_sync (bool) – avoid fsync calls if specified
warehousepg_dbid (int) – the WarehousePG database id
args (List[str]) – additional arguments

_get_compression_args(version, compression)View on GitHub #

Determine compression related arguments for pg_basebackup from the supplied compression options in the format required by the pg_basebackup version.

Parameters:

version (Version) – The pg_basebackup version for which the arguments should be formatted.
compression (barman.compression.PgBaseBackupCompression) – the pg_basebackup compression options used for this backup

class barman.command_wrappers.PgCombineBackup(destination, command, tbs_mapping=None, copy_mode=None, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_combinebackup system command

COMMAND_ALTERNATIVES = ['pg_combinebackup']#

This is a list of command names to be used to find the installed command.

__init__(destination, command, tbs_mapping=None, copy_mode=None, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

destination (str) – destination directory path
command (str) – the command to use
tbs_mapping (None|Dict[str, str]) – used for tablespace
copy_mode (None|str) – the copy mode to use, valid options are copy, link, clone or copy-file-range
connection (PostgreSQL) – an object representing a database connection
version (Version) – the command version
app_name (str) – the application name to use for the connection
check (bool) – check if the return value is in the list of allowed values of the Command obj
args (None|List[str]) – additional arguments

class barman.command_wrappers.PgReceiveXlog(connection, destination, command, version=None, app_name=None, synchronous=False, check=True, slot_name=None, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for pg_receivexlog

COMMAND_ALTERNATIVES = ['pg_receivewal', 'pg_receivexlog']#

This is a list of command names to be used to find the installed command.

__init__(connection, destination, command, version=None, app_name=None, synchronous=False, check=True, slot_name=None, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
destination (str) – destination directory path
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
synchronous (bool) – request synchronous WAL streaming
check (bool) – check if the return value is in the list of allowed values of the Command obj
slot_name (str) – the replication slot name to use for the connection
args (List[str]) – additional arguments

class barman.command_wrappers.PgVerifyBackup(data_path, command, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Bases: PostgreSQLClient

Wrapper class for the pg_verify system command

COMMAND_ALTERNATIVES = ['pg_verifybackup']#

This is a list of command names to be used to find the installed command.

__init__(data_path, command, connection=None, version=None, app_name=None, check=True, args=None, **kwargs)View on GitHub #

Constructor

Parameters:

data_path (str) – backup data directory
command (str) – the command to use
connection (PostgreSQL) – an object representing a database connection
version (Version) – the command version
app_name (str) – the application name to use for the connection
check (bool) – check if the return value is in the list of allowed values of the Command obj
args (List[str]) – additional arguments

class barman.command_wrappers.PostgreSQLClient(connection, command, version=None, app_name=None, path=None, **kwargs)View on GitHub #

Bases: Command

Superclass of all the PostgreSQL client commands.

COMMAND_ALTERNATIVES = None#

This is a list of command names to be used to find the installed command.

__init__(connection, command, version=None, app_name=None, path=None, **kwargs)View on GitHub #

Constructor

Parameters:

connection (PostgreSQL) – an object representing a database connection
command (str) – the command to use
version (Version) – the command version
app_name (str) – the application name to use for the connection
path (str) – additional path for executable retrieval

classmethod find_command(path=None)View on GitHub #

Find the active command, given all the alternatives as set in the property named COMMAND_ALTERNATIVES in this class.

Parameters:: path (str) – The path to use while searching for the command
Return type:: Command

classmethod get_version_info(path=None)View on GitHub #

Return a dictionary containing all the info about the version of the PostgreSQL client

Parameters:: path (str) – the PATH env

class barman.command_wrappers.Rsync(rsync='rsync', args=None, ssh=None, ssh_options=None, bwlimit=None, exclude=None, exclude_and_protect=None, include=None, network_compression=None, path=None, **kwargs)View on GitHub #

Bases: Command

This class is a wrapper for the rsync system command, which is used vastly by barman

__init__(rsync='rsync', args=None, ssh=None, ssh_options=None, bwlimit=None, exclude=None, exclude_and_protect=None, include=None, network_compression=None, path=None, **kwargs)View on GitHub #

Parameters:

rsync (str) – rsync executable name
args (list[str]|None) – List of additional argument to always append
ssh (str) – the ssh executable to be used when building the -e argument
ssh_options (list[str]) – the ssh options to be used when building the -e argument
bwlimit (str) – optional bandwidth limit
exclude (list[str]) – list of file to be excluded from the copy
exclude_and_protect (list[str]) – list of file to be excluded from the copy, preserving the destination if exists
include (list[str]) – list of files to be included in the copy even if excluded.
network_compression (bool) – enable the network compression
path (str) – PATH to be used while searching for cmd
check (bool) – Raise a CommandFailedException if the exit code is not present in allowed_retval
allowed_retval (list[int]) – List of exit codes considered as a successful termination.

_args_for_suse(args)View on GitHub #

Mangle args for SUSE compatibility

See https://bugzilla.opensuse.org/show_bug.cgi?id=898513

from_file_list(filelist, src, dst, *args, **kwargs)View on GitHub #

This method copies filelist from src to dst.

Returns the return code of the rsync command

get_output(*args, **kwargs)View on GitHub #: Run the command and return the output and the error (if present)

class barman.command_wrappers.RsyncPgData(rsync='rsync', args=None, **kwargs)View on GitHub #

Bases: Rsync

This class is a wrapper for rsync, specialised in sync-ing the Postgres data directory

__init__(rsync='rsync', args=None, **kwargs)View on GitHub #

Constructor

Parameters:: rsync (str) – command to run

class barman.command_wrappers.StreamLineProcessor(fobject, handler)View on GitHub #

Bases: object

Class deputed to reading lines from a file object, using a buffered read.

NOTE: This class never call os.read() twice in a row. And is designed to work with the select.select() method.

__init__(fobject, handler)View on GitHub #

Parameters:

fobject (file) – The file that is being read
handler (callable) – The function (taking only one unicode string argument) which will be called for every line

fileno()View on GitHub #

Method used by select.select() to get the underlying file descriptor.

Return type:: the underlying file descriptor

process()View on GitHub #

Read the ready data from the stream and for each line found invoke the handler.

Return bool:: True when End Of File has been reached

barman.command_wrappers.full_command_quote(command, args=None)View on GitHub #

Produce a command with quoted arguments

Parameters:

command (str) – the command to be executed
args (list[str]) – the command arguments

Return type:

str

barman.command_wrappers.shell_quote(arg)View on GitHub #

Quote a string argument to be safely included in a shell command line.

Parameters:: arg (str) – The script argument
Returns:: The argument quoted

barman.command_wrappers module#

This Page