barman.backup_executor module#

Backup Executor module

A Backup Executor is a class responsible for the execution of a backup. Specific implementations of backups are defined by classes that derive from BackupExecutor (e.g.: backup with rsync through Ssh).

A BackupExecutor is invoked by the BackupManager for backup operations.

class barman.backup_executor.BackupExecutor(backup_manager, mode=None)View on GitHub#

Bases: RemoteStatusMixin

Abstract base class for any backup executors.

__init__(backup_manager, mode=None)View on GitHub#

Base constructor

Parameters:

  • backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the executor

  • mode (str) – The mode used by the executor for the backup.

_abc_impl = <_abc._abc_data object>#
_purge_unused_wal_files(backup_info)View on GitHub#

It the provided backup is the first, purge all WAL files before the backup start.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – the backup to check

_start_backup_copy_message(backup_info)View on GitHub#

Output message for backup start

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

_stop_backup_copy_message(backup_info)View on GitHub#

Output message for backup end

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

abstract backup(backup_info)View on GitHub#

Perform a backup for the server - invoked by BackupManager.backup()

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

check(check_strategy)View on GitHub#

Perform additional checks - invoked by BackupManager.check()

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

fetch_remote_status()View on GitHub#

Get additional remote status info - invoked by BackupManager.get_remote_status()

This method does not raise any exception in case of errors, but set the missing values to None in the resulting dictionary.

Return type:

dict[str, None|str]

init()View on GitHub#

Initialise the internal state of the backup executor

property modeView on GitHub#

Property that defines the mode used for the backup.

If a strategy is present, the returned string is a combination of the mode of the executor and the mode of the strategy (eg: rsync-exclusive)

Return str:

a string describing the mode used for the backup

status()View on GitHub#

Set additional status info - invoked by BackupManager.status()

class barman.backup_executor.BackupStrategy(postgres, server_name, mode=None)View on GitHub#

Bases: object

Abstract base class for a strategy to be used by a backup executor.

START_TIME_RE = re.compile('^START TIME: (.*)', re.MULTILINE)#

Regex for START WAL LOCATION info

WAL_RE = re.compile('^START WAL LOCATION: (.*) \\(file (.*)\\)', re.MULTILINE)#

Regex for START TIME info

__init__(postgres, server_name, mode=None)View on GitHub#

Constructor

Parameters:
_abc_impl = <_abc._abc_data object>#
_backup_info_from_backup_label(backup_info)View on GitHub#

Fill a backup info with information from the backup_label file

Parameters:

backup_info (barman.infofile.BackupInfo) – object representing a backup

static _backup_info_from_start_location(backup_info, start_info)View on GitHub#

Fill a backup info with information from a start_backup

Parameters:

backup_info (barman.infofile.BackupInfo) – object

representing a

backup

Parameters:

start_info (DictCursor) – the result of the pg_backup_start

command

static _backup_info_from_stop_location(backup_info, stop_info)View on GitHub#

Fill a backup info with information from a backup stop location

Parameters:

  • backup_info (barman.infofile.BackupInfo) – object representing a backup

  • stop_info (DictCursor) – location info of stop backup

_pg_get_metadata(backup_info)View on GitHub#

Load PostgreSQL metadata into the backup_info parameter

Parameters:

backup_info (barman.infofile.BackupInfo) – backup information

_read_backup_label(backup_info)View on GitHub#

Read the backup_label file

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

abstract check(check_strategy)View on GitHub#

Perform additional checks - invoked by BackupExecutor.check()

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

start_backup(backup_info)View on GitHub#

Issue a start of a backup - invoked by BackupExecutor.backup()

Parameters:

backup_info (barman.infofile.BackupInfo) – backup information

status()View on GitHub#

Set additional status info - invoked by BackupExecutor.status()

abstract stop_backup(backup_info)View on GitHub#

Issue a stop of a backup - invoked by BackupExecutor.backup()

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

class barman.backup_executor.ConcurrentBackupStrategy(postgres, server_name)View on GitHub#

Bases: BackupStrategy

Concrete class for concurrent backup strategy.

This strategy is responsible for coordinating Barman with PostgreSQL on concurrent physical backup operations through concurrent backup PostgreSQL api.

__init__(postgres, server_name)View on GitHub#

Constructor

Parameters:
_abc_impl = <_abc._abc_data object>#
_concurrent_start_backup(backup_info, label)View on GitHub#

Start a concurrent backup using the PostgreSQL 9.6 concurrent backup api

Parameters:
_concurrent_stop_backup(backup_info)View on GitHub#

Stop a concurrent backup using the PostgreSQL 9.6 concurrent backup api

Parameters:

backup_info (barman.infofile.BackupInfo) – backup information

check(check_strategy)View on GitHub#

Checks that Postgres is at least minimal version

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

start_backup(backup_info)View on GitHub#

Start of the backup.

The method performs all the preliminary operations required for a backup to start.

Parameters:

backup_info (barman.infofile.BackupInfo) – backup information

stop_backup(backup_info)View on GitHub#

Stop backup wrapper

Parameters:

backup_info (barman.infofile.BackupInfo) – backup information

class barman.backup_executor.ExclusiveBackupStrategy(postgres, server_name)View on GitHub#

Bases: BackupStrategy

Concrete class for exclusive backup strategy.

This strategy is for ExternalBackupExecutor only and is responsible for coordinating Barman with PostgreSQL on standard physical backup operations (known as ‘exclusive’ backup), such as invoking pg_start_backup() and pg_stop_backup() on the master server.

__init__(postgres, server_name)View on GitHub#

Constructor

Parameters:
_abc_impl = <_abc._abc_data object>#
check(check_strategy)View on GitHub#

Perform additional checks for ExclusiveBackupStrategy

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

start_backup(backup_info)View on GitHub#

Manage the start of an exclusive backup

The method performs all the preliminary operations required for an exclusive physical backup to start, as well as preparing the information on the backup for Barman.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

stop_backup(backup_info)View on GitHub#

Manage the stop of an exclusive backup

The method informs the PostgreSQL server that the physical exclusive backup is finished, as well as preparing the information returned by PostgreSQL for Barman.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

class barman.backup_executor.ExternalBackupExecutor(backup_manager, mode, local_mode=False)View on GitHub#

Bases: BackupExecutor

Abstract base class for non-postgres backup executors.

An external backup executor is any backup executor which uses the PostgreSQL low-level backup API to coordinate the backup.

Such executors can operate remotely via SSH or locally:

  • remote mode (default), operates via SSH

  • local mode, operates as the same user that Barman runs with

It is also a factory for exclusive/concurrent backup strategy objects.

Raises a SshCommandException if ‘ssh_command’ is not set and not operating in local mode.

__init__(backup_manager, mode, local_mode=False)View on GitHub#

Constructor of the abstract class for backups via Ssh

Parameters:

  • backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the executor

  • mode (str) – The mode used by the executor for the backup.

  • local_mode (bool) – if set to False (default), the class is able to operate on remote servers using SSH. Operates only locally if set to True.

_abc_impl = <_abc._abc_data object>#
_local_check(check_strategy)View on GitHub#

Specific checks for local mode of ExternalBackupExecutor (same user)

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

_remote_check(check_strategy)View on GitHub#

Specific checks for remote mode of ExternalBackupExecutor, via SSH.

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

_update_action_from_strategy()View on GitHub#

Update the executor’s current action with the one of the strategy. This is used during exception handling to let the caller know where the failure occurred.

backup(backup_info)View on GitHub#

Perform a backup for the server - invoked by BackupManager.backup() through the generic interface of a BackupExecutor. This implementation is responsible for performing a backup through a remote connection to the PostgreSQL server via Ssh. The specific set of instructions depends on both the specific class that derives from ExternalBackupExecutor and the selected strategy (e.g. exclusive backup through Rsync).

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

abstract backup_copy(backup_info)View on GitHub#

Performs the actual copy of a backup for the server

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

check(check_strategy)View on GitHub#

Perform additional checks for ExternalBackupExecutor, including Ssh connection (executing a ‘true’ command on the remote server) and specific checks for the given backup strategy.

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

fetch_remote_status()View on GitHub#

Get remote information on PostgreSQL using Ssh, such as last archived WAL file

This method does not raise any exception in case of errors, but set the missing values to None in the resulting dictionary.

Return type:

dict[str, None|str]

status()View on GitHub#

Set additional status info for ExternalBackupExecutor using remote commands via Ssh, as well as those defined by the given backup strategy.

class barman.backup_executor.LocalConcurrentBackupStrategy(postgres, server_name)View on GitHub#

Bases: ConcurrentBackupStrategy

Concrete class for concurrent backup strategy writing data locally.

This strategy is for ExternalBackupExecutor only and is responsible for coordinating Barman with PostgreSQL on concurrent physical backup operations through concurrent backup PostgreSQL api.

_abc_impl = <_abc._abc_data object>#
_write_backup_label(backup_info)View on GitHub#

Write the backup_label file inside local data directory

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

stop_backup(backup_info)View on GitHub#

Stop backup wrapper

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

class barman.backup_executor.PassiveBackupExecutor(backup_manager)View on GitHub#

Bases: BackupExecutor

Dummy backup executors for Passive servers.

Raises a SshCommandException if ‘primary_ssh_command’ is not set.

__init__(backup_manager)View on GitHub#

Constructor of Dummy backup executors for Passive servers.

Parameters:

backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the executor

_abc_impl = <_abc._abc_data object>#
backup(backup_info)View on GitHub#

This method should never be called, because this is a passive server

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

check(check_strategy)View on GitHub#

Perform additional checks for PassiveBackupExecutor, including Ssh connection to the primary (executing a ‘true’ command on the remote server).

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

property modeView on GitHub#

Property that defines the mode used for the backup. :return str: a string describing the mode used for the backup

status()View on GitHub#

Set additional status info for PassiveBackupExecutor.

class barman.backup_executor.PostgresBackupExecutor(backup_manager)View on GitHub#

Bases: BackupExecutor

Concrete class for backup via pg_basebackup (plain format).

Relies on pg_basebackup command to copy data files from the PostgreSQL cluster using replication protocol.

__init__(backup_manager)View on GitHub#

Constructor

Parameters:

backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the executor

_abc_impl = <_abc._abc_data object>#
_err_handler(line)View on GitHub#

Handler invoked during a backup when anything is sent to stderr.

Used to perform a WAL switch on a primary server if pg_basebackup is running against a standby, otherwise just logs output at INFO level.

Parameters:

line (str) – The error line to be handled.

_prepare_backup_destination(dest_dirs)View on GitHub#

Prepare the destination of the backup, including tablespaces.

This method is also responsible for removing a directory if it already exists and for ensuring the correct permissions for the created directories

Parameters:

dest_dirs (list[str]) – destination directories

_retry_handler(dest_dirs, command, args, kwargs, attempt, exc)View on GitHub#

Handler invoked during a backup in case of retry.

The method simply warn the user of the failure and remove the already existing directories of the backup.

Parameters:

  • dest_dirs (list[str]) – destination directories

  • command (RsyncPgData) – Command object being executed

  • args (list) – command args

  • kwargs (dict) – command kwargs

  • attempt (int) – attempt number (starting from 0)

  • exc (CommandFailedException) – the exception which caused the failure

_start_backup_copy_message(backup_info)View on GitHub#

Output message for backup start

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

_validate_compression(remote_status)View on GitHub#

In charge of validating compression options.

Note: Because this method requires a connection to the PostgreSQL server it should be called within the context of a closing context manager.

Parameters:

remote_status

Returns:

backup(backup_info)View on GitHub#

Perform a backup for the server - invoked by BackupManager.backup() through the generic interface of a BackupExecutor.

This implementation is responsible for performing a backup through the streaming protocol.

The connection must be made with a superuser or a user having REPLICATION permissions (see PostgreSQL documentation, Section 20.2), and pg_hba.conf must explicitly permit the replication connection. The server must also be configured with enough max_wal_senders to leave at least one session available for the backup.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

backup_copy(backup_info)View on GitHub#

Perform the actual copy of the backup using pg_basebackup. First, manages tablespaces, then copies the base backup using the streaming protocol.

In case of failure during the execution of the pg_basebackup command the method raises a DataTransferFailure, this trigger the retrying mechanism when necessary.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

check(check_strategy)View on GitHub#

Perform additional checks for PostgresBackupExecutor

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

fetch_remote_status()View on GitHub#

Gather info from the remote server.

This method does not raise any exception in case of errors, but set the missing values to None in the resulting dictionary.

validate_configuration()View on GitHub#

Validate the configuration for this backup executor.

If the configuration is not compatible this method will disable the server.

class barman.backup_executor.PostgresBackupStrategy(postgres, server_name, backup_compression=None)View on GitHub#

Bases: BackupStrategy

Concrete class for postgres backup strategy.

This strategy is for PostgresBackupExecutor only and is responsible for executing pre e post backup operations during a physical backup executed using pg_basebackup.

__init__(postgres, server_name, backup_compression=None)View on GitHub#

Constructor

Parameters:
_abc_impl = <_abc._abc_data object>#
_read_backup_label(backup_info)View on GitHub#

Read the backup_label file.

Transparently handles the fact that the backup_label file may be in a compressed tarball.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

_read_compressed_backup_label(backup_info)View on GitHub#

Read the contents of a backup_label file from a compressed archive.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

check(check_strategy)View on GitHub#

Perform additional checks for the Postgres backup strategy

start_backup(backup_info)View on GitHub#

Manage the start of an pg_basebackup backup

The method performs all the preliminary operations required for a backup executed using pg_basebackup to start, gathering information from postgres and filling the backup_info.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

stop_backup(backup_info)View on GitHub#

Manage the stop of an pg_basebackup backup

The method retrieves the information necessary for the backup.info file reading the backup_label file.

Due of the nature of the pg_basebackup, information that are gathered during the start of a backup performed using rsync, are retrieved here

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

class barman.backup_executor.RsyncBackupExecutor(backup_manager, local_mode=False)View on GitHub#

Bases: ExternalBackupExecutor

Concrete class for backup via Rsync+Ssh.

It invokes PostgreSQL commands to start and stop the backup, depending on the defined strategy. Data files are copied using Rsync via Ssh. It heavily relies on methods defined in the ExternalBackupExecutor class from which it derives.

__init__(backup_manager, local_mode=False)View on GitHub#

Constructor

Parameters:

backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the strategy

_abc_impl = <_abc._abc_data object>#
_format_src(path)View on GitHub#

If the executor is operating in remote mode, add a : in front of the path for rsync to work via SSH.

Parameters:

path (string) – the path to format

Return str:

the formatted path string

_reuse_path(previous_backup_info, tablespace=None)View on GitHub#

If reuse_backup is ‘copy’ or ‘link’, builds the path of the directory to reuse, otherwise always returns None.

If oid is None, it returns the full path of PGDATA directory of the previous_backup otherwise it returns the path to the specified tablespace using it’s oid.

Parameters:
Returns:

a string containing the local path with data to be reused or None

Return type:

str|None

_start_backup_copy_message(backup_info)View on GitHub#

Output message for backup start.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

backup(*args, **kwargs)View on GitHub#

Perform an Rsync backup.

Note

This method currently only calls the parent backup method but inside a keepalive context to ensure the connection does not become idle long enough to get dropped by a firewall, for instance. This is important to ensure that pg_backup_start() and pg_backup_stop() are called within the same session.

backup_copy(backup_info)View on GitHub#

Perform the actual copy of the backup using Rsync.

First, it copies one tablespace at a time, then the PGDATA directory, and finally configuration files (if outside PGDATA). Bandwidth limitation, according to configuration, is applied in the process. This method is the core of base backup copy using Rsync+Ssh.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – backup information

validate_configuration()View on GitHub#
class barman.backup_executor.SnapshotBackupExecutor(backup_manager)View on GitHub#

Bases: ExternalBackupExecutor

Concrete class which uses cloud provider disk snapshots to create backups.

It invokes PostgreSQL commands to start and stop the backup, depending on the defined strategy.

It heavily relies on methods defined in the ExternalBackupExecutor class from which it derives.

No data files are copied and instead snapshots are created of the requested disks using the cloud provider API (abstracted through a CloudSnapshotInterface).

As well as ensuring the backup happens via snapshot copy, this class also:

  • Checks that the specified disks are attached to the named instance.

  • Checks that the specified disks are mounted on the named instance.

  • Records the mount points and options of each disk in the backup info.

Barman will still store the following files in its backup directory:

  • The backup_label (for concurrent backups) which is written by the LocalConcurrentBackupStrategy.

  • The backup.info which is written by the BackupManager responsible for instantiating this class.

__init__(backup_manager)View on GitHub#

Constructor for the SnapshotBackupExecutor

Parameters:

backup_manager (barman.backup.BackupManager) – the BackupManager assigned to the strategy

_abc_impl = <_abc._abc_data object>#
_start_backup_copy_message(backup_info)View on GitHub#

Output message for backup start.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – Backup information.

_stop_backup_copy_message(backup_info)View on GitHub#

Output message for backup end.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – Backup information.

_validate_aws_lock_configuration()View on GitHub#

Verify configuration is valid for locking a snapshot backup using AWS provider.

static add_mount_data_to_volume_metadata(volumes, remote_cmd)View on GitHub#

Adds the mount point and mount options for each supplied volume.

Calls resolve_mounted_volume on each supplied volume so that the volume metadata (which originated from the cloud provider) can be resolved to the mount point and mount options of the volume as mounted on a compute instance.

This will set the current mount point and mount options of the volume so that they can be stored in the snapshot metadata for the backup when the backup is taken.

Parameters:

  • volumes (dict[str,barman.cloud.VolumeMetadata]) – Metadata for the volumes attached to a specific compute instance.

  • remote_cmd (UnixLocalCommand) – Wrapper for executing local/remote commands on the compute instance to which the volumes are attached.

backup_copy(backup_info)View on GitHub#

Perform the backup using cloud provider disk snapshots.

Parameters:

backup_info (barman.infofile.LocalBackupInfo) – Backup information.

check(check_strategy)View on GitHub#

Perform additional checks for SnapshotBackupExecutor, specifically:

  • check that the VM instance for which snapshots should be taken exists

  • check that the expected disks are attached to that instance

  • check that the attached disks are mounted on the filesystem

Parameters:

check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks

static find_missing_and_unmounted_disks(cmd, snapshot_interface, snapshot_instance, snapshot_disks)View on GitHub#

Checks for any disks listed in snapshot_disks which are not correctly attached and mounted on the named instance and returns them as a tuple of two lists.

This is used for checking that the disks which are to be used as sources for snapshots at backup time are attached and mounted on the instance to be backed up.

Parameters:

  • cmd (UnixLocalCommand) – Wrapper for local/remote commands.

  • snapshot_interface (barman.cloud.CloudSnapshotInterface) – Interface for taking snapshots and associated operations via cloud provider APIs.

  • snapshot_instance (str) – The name of the VM instance to which the disks to be backed up are attached.

  • snapshot_disks (list[str]) – A list containing the names of the disks for which snapshots should be taken at backup time.

:rtype tuple[list[str],list[str]] :return: A tuple where the first element is a list of all disks which are not

attached to the VM instance and the second element is a list of all disks which are attached but not mounted.

validate_configuration()View on GitHub#

Verify configuration is valid for a snapshot backup.

barman.backup_executor._parse_ssh_command(ssh_command)View on GitHub#

Parse a user provided ssh command to a single command and a list of arguments

In case of error, the first member of the result (the command) will be None

Parameters:

ssh_command – a ssh command provided by the user

Return tuple[str,list[str]]:

the command and a list of options

On this page
Edit on GitHub
Show Source