barman.server module#
This module represents a Server. Barman is able to manage multiple servers.
- class barman.server.CheckOutputStrategyView on GitHub#
Bases:
CheckStrategy
This strategy for the ‘check’ command immediately sends the result of a check to the designated output channel. This class derives from the basic CheckStrategy, reuses the same logic and adds output messages.
- __init__()View on GitHub#
Output Strategy constructor
- result(server_name, status, hint=None, check=None, perfdata=None)View on GitHub#
Store the result of a check. Log any check result (error or debug level). Output the result to the user
- class barman.server.CheckStrategy(ignore_checks=['minimum redundancy requirements', 'backup maximum age', 'backup minimum size', 'failed backups', 'archiver errors', 'empty incoming directory', 'empty streaming directory', 'incoming WALs directory', 'streaming WALs directory', 'wal maximum age', 'PostgreSQL server is standby'])View on GitHub#
Bases:
object
This strategy for the ‘check’ collects the results of every check and does not print any message. This basic class is also responsible for immediately logging any performed check with an error in case of check failure and a debug message in case of success.
- class CheckResult(server_name, check, status)#
Bases:
tuple
- _asdict()#
Return a new dict which maps field names to their values.
- _field_defaults = {}#
- _fields = ('server_name', 'check', 'status')#
- classmethod _make(iterable)#
Make a new CheckResult object from a sequence or iterable
- _replace(**kwds)#
Return a new CheckResult object replacing specified fields with new values
- check#
Alias for field number 1
- server_name#
Alias for field number 0
- status#
Alias for field number 2
- NON_CRITICAL_CHECKS = ['minimum redundancy requirements', 'backup maximum age', 'backup minimum size', 'failed backups', 'archiver errors', 'empty incoming directory', 'empty streaming directory', 'incoming WALs directory', 'streaming WALs directory', 'wal maximum age', 'PostgreSQL server is standby']#
- __init__(ignore_checks=['minimum redundancy requirements', 'backup maximum age', 'backup minimum size', 'failed backups', 'archiver errors', 'empty incoming directory', 'empty streaming directory', 'incoming WALs directory', 'streaming WALs directory', 'wal maximum age', 'PostgreSQL server is standby'])View on GitHub#
Silent Strategy constructor
- Parameters:
ignore_checks (list) – list of checks that can be ignored
- _check_name(check)View on GitHub#
- init_check(check_name)View on GitHub#
Mark in the debug log when barman starts the execution of a check
- Parameters:
check_name (str) – the name of the check that is starting
- result(server_name, status, hint=None, check=None, perfdata=None)View on GitHub#
Store the result of a check (with no output). Log any check result (error or debug level).
- class barman.server.Server(config)View on GitHub#
Bases:
RemoteStatusMixin
This class represents the PostgreSQL server to backup.
- XLOG_DB = 'xlog.db'#
- __default_check_strategy = <barman.server.CheckOutputStrategy object>#
- __init__(config)View on GitHub#
Server constructor.
- Parameters:
config (barman.config.ServerConfig) – the server configuration
- _abc_impl = <_abc._abc_data object>#
- static _build_path(path_prefix=None)View on GitHub#
If a path_prefix is provided build a string suitable to be used in PATH environment variable by joining the path_prefix with the current content of PATH environment variable.
If the path_prefix is None returns None.
- Return type:
str|None
- _check_has_monitoring_privileges(check_strategy, remote_status, suffix=None)View on GitHub#
Check whether the remote status indicates monitoring information can be read.
- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
suffix (str|None) – A suffix to be appended to the check name
- _check_replication_slot(check_strategy, remote_status, suffix=None)View on GitHub#
Check the replication slot used for WAL streaming.
If
streaming_archiver
is enabled, checks that the replication slot specified in the configuration exists, is initialised and is active.If
streaming_archiver
is disabled, checks that the replication slot does not exist.- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
suffix (str|None) – A suffix to be appended to the check name
- _check_standby(check_strategy)View on GitHub#
Perform checks specific to a primary/standby configuration.
- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the results of the various checks.
- _check_streaming_supported(check_strategy, remote_status, suffix=None)View on GitHub#
Check whether the remote status indicates streaming is possible.
- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
suffix (str|None) – A suffix to be appended to the check name
- _check_wal_info(wal_info, last_wal_maximum_age)View on GitHub#
Checks the supplied wal_info is within the last_wal_maximum_age.
- Parameters:
last_backup_minimum_age – timedelta representing the time from now during which a WAL is considered valid
- Return tuple:
a tuple containing the boolean result of the check, a string with auxiliary information about the check, and an integer representing the size of the WAL in bytes
- _check_wal_level(check_strategy, remote_status, suffix=None)View on GitHub#
Check whether the remote status indicates
wal_level
is correct.- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
suffix (str|None) – A suffix to be appended to the check name
- _check_wal_queue(check_strategy, dir_name, archiver_name)View on GitHub#
Check if one of the wal queue directories beyond the max file threshold
- _check_wal_streaming_preflight(check_strategy, remote_status)View on GitHub#
Verify the supplied remote_status indicates WAL streaming is possible.
Uses the remote status information to run the
_check_streaming_supported()
,_check_wal_level()
andcheck_identity()
checks in order to verify that the connections can be used for WAL streaming. Also runs an additional_has_monitoring_privileges()
check, which validates the WAL-specific conninfo connects with a user than can read monitoring information.- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
- _init_archivers()View on GitHub#
- _init_bandwidth_limits()View on GitHub#
- _init_minimum_redundancy()View on GitHub#
- _init_postgres(config)View on GitHub#
- _init_retention_policies()View on GitHub#
- _make_directories()View on GitHub#
Make backup directories in case they do not exist
- archive_wal(verbose=True)View on GitHub#
Perform the WAL archiving operations.
Usually run as subprocess of the barman cron command, but can be executed manually using the barman archive-wal command
- Parameters:
verbose (bool) – if false outputs something only if there is at least one file
- background_receive_wal(keep_descriptors)View on GitHub#
Method that handles the start of a ‘receive-wal’ sub process, running in background.
This method must be run protected by ServerCronLock :param bool keep_descriptors: whether to keep subprocess
descriptors attached to this process.
- backup(wait=False, wait_timeout=None, backup_name=None, **kwargs)View on GitHub#
Performs a backup for the server :param bool wait: wait for all the required WAL files to be archived :param int|None wait_timeout: the time, in seconds, the backup
will wait for the required WAL files to be archived before timing out
- check(check_strategy=<barman.server.CheckOutputStrategy object>)View on GitHub#
Implements the ‘server check’ command and makes sure SSH and PostgreSQL connections work properly. It checks also that backup directories exist (and if not, it creates them).
The check command will time out after a time interval defined by the check_timeout configuration value (default 30 seconds)
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_archive(check_strategy)View on GitHub#
Checks WAL archive
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_archiver_errors(check_strategy)View on GitHub#
Checks the presence of archiving errors
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the check
- check_backup(backup_info)View on GitHub#
Make sure that we have all the WAL files required by a physical backup for consistency (from the first to the last WAL file)
- Parameters:
backup_info – the target backup
- check_backup_validity(check_strategy)View on GitHub#
Check if backup validity requirements are satisfied
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_configuration(check_strategy)View on GitHub#
Check for error messages in the message list of the server and output eventual errors
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_directories(check_strategy)View on GitHub#
Checks backup directories and creates them if they do not exist
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_identity(check_strategy, remote_status=None, suffix=None)View on GitHub#
Check the systemid retrieved from the streaming connection is the same that is retrieved from the standard connection, and then verifies it matches the one stored on disk.
- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
remote_status (dict[str, None|str]) – Remote status information used by this check
suffix (str|None) – A suffix to be appended to the check name
- check_postgres(check_strategy)View on GitHub#
Checks PostgreSQL connection
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_retention_policy_settings(check_strategy)View on GitHub#
Checks retention policy setting
- Parameters:
check_strategy (CheckStrategy) – the strategy for the management of the results of the various checks
- check_sync_required(backup_name, primary_info, local_backup_info)View on GitHub#
Check if it is necessary to sync a backup.
If the backup is present on the Primary node:
if it does not exist locally: continue (synchronise it)
if it exists and is DONE locally: raise SyncNothingToDo (nothing to do)
if it exists and is FAILED locally: continue (try to recover it)
If the backup is not present on the Primary node:
if it does not exist locally: raise SyncError (wrong call)
if it exists and is DONE locally: raise SyncNothingToDo (nothing to do)
if it exists and is FAILED locally: raise SyncToBeDeleted (remove it)
If a backup needs to be synchronised but it is obsolete according to local retention policies, raise SyncNothingToDo, else return to the caller.
- Parameters:
backup_name (str) – str name of the backup to sync
primary_info (dict) – dict containing the Primary node status
local_backup_info (barman.infofile.BackupInfo) – BackupInfo object representing the current backup state
- Raises:
SyncError – There is an error in the user request
SyncNothingToDo – Nothing to do for this request
SyncToBeDeleted – Backup is not recoverable and must be deleted
- check_wal_streaming(check_strategy)View on GitHub#
Perform checks related to the streaming of WALs only (not backups).
If no WAL-specific connection information is defined then checks already performed on the default connection information will have verified their suitability for WAL streaming so this check will only call
_check_replication_slot()
for the existing streaming connection as this is the only additional check required.If WAL-specific connection information is defined then we must verify that streaming is possible using that connection information as well as check the replication slot. This check will therefore:
Create these connections.
Fetch the remote status of these connections.
Pass the remote status information to
_check_wal_streaming_preflight()
which will verify that the status information returned by these connections indicates they are suitable for WAL streaming.Pass the remote status information to
_check_replication_slot()
so that the status of the replication slot can be verified.
- Parameters:
check_strategy (CheckStrategy) – The strategy for the management of the result of this check
- check_wal_validity(check_strategy)View on GitHub#
Check if wal archiving requirements are satisfied
- close()View on GitHub#
Close all the open connections to PostgreSQL
- create_physical_repslot()View on GitHub#
Create a physical replication slot using the streaming connection
- cron(wals=True, retention_policies=True, keep_descriptors=False)View on GitHub#
Maintenance operations
- cron_archive_wal(keep_descriptors)View on GitHub#
Method that handles the start of an ‘archive-wal’ sub-process.
This method must be run protected by ServerCronLock :param bool keep_descriptors: whether to keep subprocess descriptors
attached to this process.
- cron_check_backup(keep_descriptors)View on GitHub#
Method that handles the start of a ‘check-backup’ sub process
- Parameters:
keep_descriptors (bool) – whether to keep subprocess descriptors attached to this process.
- delete_backup(backup)View on GitHub#
Deletes a backup.
Performs some checks to confirm that the backup can indeed be deleted and if so it is deleted along with all backups that depend on it, if any.
- Parameters:
backup (barman.infofile.LocalBackupInfo) – the backup to delete
- Return bool:
True if deleted, False if could not delete the backup
- drop_repslot()View on GitHub#
Drop a replication slot using the streaming connection
- fetch_remote_status()View on GitHub#
Get the status of the remote server
This method does not raise any exception in case of errors, but set the missing values to None in the resulting dictionary.
- get_available_backups(status_filter=('WAITING_FOR_WALS', 'DONE'))View on GitHub#
Get a list of available backups
- param: status_filter: the status of backups to return,
default to BackupManager.DEFAULT_STATUS_FILTER
- get_backup(backup_id)View on GitHub#
Return the backup information for the given backup id.
If the backup_id is None or backup.info file doesn’t exists, it returns None.
- Parameters:
backup_id (str|None) – the ID of the backup to return
- Return type:
- get_backup_ext_info(backup_info)View on GitHub#
Return a dictionary containing all available information about a backup
The result is equivalent to the sum of information from
BackupInfo object
the Server.get_wal_info() return value
the context in the catalog (if available)
the retention policy status
the copy statistics
the incremental backups information
extra backup.info properties
- Parameters:
backup_info – the target backup
- Rtype dict:
all information about a backup
- get_backup_id_from_name(backup_name, status_filter=('WAITING_FOR_WALS', 'DONE'))View on GitHub#
Get the id of the named backup, if it exists.
- Parameters:
backup_name (string) – The name of the backup for which an ID should be returned
status_filter (tuple) – The status of the backup to return.
- Return string|None:
ID of the backup
- get_children_timelines(tli, forked_after=None)View on GitHub#
Get a list of the children of the passed timeline
- get_first_backup_id(status_filter=('WAITING_FOR_WALS', 'DONE'))View on GitHub#
Get the id of the oldest/first backup in the catalog (if exists)
- Parameters:
status_filter – The status of the backup to return, default to DEFAULT_STATUS_FILTER.
- Return string|None:
ID of the backup
- get_identity_file_path()View on GitHub#
Get the path of the file that should contain the identity of the cluster :rtype: str
- get_last_backup_id(status_filter=('WAITING_FOR_WALS', 'DONE'))View on GitHub#
Get the id of the latest/last backup in the catalog (if exists)
- Parameters:
status_filter – The status of the backup to return, default to
BackupManager.DEFAULT_STATUS_FILTER
.- Return str|None:
ID of the backup
- get_last_full_backup_id(status_filter=('WAITING_FOR_WALS', 'DONE'))View on GitHub#
Get the id of the latest/last FULL backup in the catalog (if exists)
- Parameters:
status_filter – The status of the backup to return, default to DEFAULT_STATUS_FILTER.
- Return string|None:
ID of the backup
- get_next_backup(backup_id)View on GitHub#
Get the next backup (if any) from the catalog
- Parameters:
backup_id – the backup id from which return the next
- get_previous_backup(backup_id)View on GitHub#
Get the previous backup (if any) from the catalog
- Parameters:
backup_id – the backup id from which return the previous
- get_required_xlog_files(backup, target_tli=None, target_time=None, target_xid=None, target_lsn=None, target_immediate=False)View on GitHub#
Get the xlog files required for a recovery.
Note
target_time and target_xid are ignored by this method. As it can be very expensive to parse WAL dumps to identify which WAL files are required to honor the specific targets, we simply copy all WAL files up to the calculated target timeline, so we make sure recovery will be able to finish successfully (assuming the archived WALs honor the specified targets).
On the other hand, target_tli, target_lsn and target_immediate are easier to handle, so we only copy the WALs required to reach the requested targets.
- Parameters:
backup (BackupInfo) – a backup object
- :param target_tlitarget timeline, either a timeline ID or one of the keywords
supported by Postgres
- Parameters:
target_time – target time, in epoch
target_xid – target transaction ID
target_lsn – target LSN
target_immediate – target that ends recovery as soon as consistency is reached. Defaults to
False
.
- get_wal(wal_name, compression=None, keep_compression=False, output_directory=None, peek=None, partial=False)View on GitHub#
Retrieve a WAL file from the archive
- Parameters:
wal_name (str) – id of the WAL file to find into the WAL archive
compression (str|None) – compression format for the output
keep_compression (bool) – if True, do not uncompress compressed WAL files
output_directory (str|None) – directory where to deposit the WAL file
peek (int|None) – if defined list the next N WAL file
partial (bool) – retrieve also partial WAL files
- get_wal_full_path(wal_name)View on GitHub#
Build the full path of a WAL for a server given the name
- Parameters:
wal_name – WAL file name
- get_wal_info(backup_info)View on GitHub#
Returns information about WALs for the given backup
- Parameters:
backup_info (barman.infofile.LocalBackupInfo) – the target backup
- get_wal_possible_paths(wal_name, partial=False)View on GitHub#
Build a list of possible positions of a WAL file
- get_wal_sendfile(wal_file, compression, keep_compression, destination)View on GitHub#
Send a WAL file to the destination file, using the required compression
- get_wal_until_next_backup(backup, include_history=False)View on GitHub#
Get the xlog files between backup and the next
- Parameters:
backup (BackupInfo) – a backup object, the starting point to retrieve WALs
include_history (bool) – option for the inclusion of include_history files into the output
- is_backup_locally_obsolete(backup_name, remote_backups)View on GitHub#
Check if a remote backup is obsolete according with the local retention policies.
- Parameters:
self (barman.server.Server) – Server object
backup_name (str) – str name of the backup to sync
remote_backups (dict) – dict containing the Primary node status
- Return bool:
returns if the backup is obsolete or not
- kill(task, fail_if_not_present=True)View on GitHub#
Given the name of a barman sub-task type, attempts to stop all the processes
- Parameters:
task (string) – The task we want to stop
fail_if_not_present (bool) – Display an error when the process is not present (default: True)
- list_backups()View on GitHub#
Lists all the available backups for the server
- load_primary_info()View on GitHub#
Load the content of PRIMARY_INFO_FILE for the given server
- Return dict:
primary server information
- load_sync_wals_info()View on GitHub#
Load the content of SYNC_WALS_INFO_FILE for the given server
- Return collections.namedtuple:
last read wal and position information
- perform_delete_backup(backup)View on GitHub#
Performs the deletion of a backup.
Deletes a single backup, ensuring that no other process can access the backup simultaneously during its deletion.
- Parameters:
backup (barman.infofile.LocalBackupInfo) – the backup to delete
- Return bool:
True if deleted, False if could not delete the backup
- primary_node_info(last_wal=None, last_position=None)View on GitHub#
Invoke sync-info directly on the specified primary node
The method issues a call to the sync-info method on the primary node through an SSH connection
- Parameters:
self (barman.server.Server) – the Server object
last_wal (str|None) – last read wal
last_position (int|None) – last read position (in xlog.db)
- Raises:
SyncError – if the ssh command fails
- put_wal(fileobj)View on GitHub#
Receive a WAL file from SERVER_NAME and securely store it in the incoming directory.
The file will be read from the fileobj passed as parameter.
- read_identity_file()View on GitHub#
Read the server identity :rtype: dict[str,str]
- rebuild_xlogdb()View on GitHub#
Rebuild the whole xlog database guessing it from the archive content.
- receive_wal(reset=False)View on GitHub#
Enable the reception of WAL files using streaming protocol.
Usually started by barman cron command. Executing this manually, the barman process will not terminate but will continuously receive WAL files from the PostgreSQL server.
- Parameters:
reset – When set, resets the status of receive-wal
- recover(backup_info, dest, tablespaces=None, remote_command=None, **kwargs)View on GitHub#
Performs a recovery of a backup
- Parameters:
backup_info (barman.infofile.LocalBackupInfo) – the backup to recover
dest (str) – the destination directory
tablespaces (dict[str,str]|None) – a tablespace name -> location map (for relocation)
remote_command (str|None) – default None. The remote command to recover the base backup, in case of remote backup.
target_tli (str|None) – the target timeline
target_time (str|None) – the target time
target_xid (str|None) – the target xid
target_lsn (str|None) – the target LSN
target_name (str|None) – the target name created previously with pg_create_restore_point() function call
target_immediate (bool|None) – end recovery as soon as consistency is reached
exclusive (bool) – whether the recovery is exclusive or not
target_action (str|None) – the recovery target action
standby_mode (bool|None) – the standby mode
recovery_conf_filename (str|None) – filename for storing recovery configurations
- replication_status(target='all')View on GitHub#
Implements the ‘replication-status’ command.
- report_backups()View on GitHub#
- restart_processes()View on GitHub#
Restart server subprocesses.
- static set_sync_starting_point(xlogdb_file, last_wal, last_position)View on GitHub#
Check if the xlog.db file has changed between two requests from the client and set the start point for reading the file
- show()View on GitHub#
Shows the server configuration
- show_backup(backup_info)View on GitHub#
Output all available information about a backup
- Parameters:
backup_info – the target backup
- status()View on GitHub#
Implements the ‘server-status’ command.
- status_postgres()View on GitHub#
Status of PostgreSQL server
- status_retention_policies()View on GitHub#
Status of retention policies enforcement
- status_wal_archiver()View on GitHub#
Status of WAL archiver(s)
- switch_wal(force=False, archive=None, archive_timeout=None)View on GitHub#
Execute the switch-wal command on the target server
- sync_backup(backup_name)View on GitHub#
Method for the synchronisation of a backup from a primary server.
The Method checks that the server is passive, then if it is possible to sync with the Primary. Acquires a lock at backup level and copy the backup from the Primary node using rsync.
During the sync process the backup on the Passive node is marked as SYNCING and if the sync fails (due to network failure, user interruption…) it is marked as FAILED.
- Parameters:
self (barman.server.Server) – the passive Server object to sync
backup_name (str) – the name of the backup to sync.
- sync_cron(keep_descriptors)View on GitHub#
Manage synchronisation operations between passive node and master node. The method recover information from the remote master server, evaluate if synchronisation with the master is required and spawn barman sub processes, syncing backups and WAL files :param bool keep_descriptors: whether to keep subprocess descriptors
attached to this process.
- sync_status(last_wal=None, last_position=None)View on GitHub#
Return server status for sync purposes.
- The method outputs JSON, containing:
list of backups (with DONE status)
server configuration
last read position (in xlog.db)
last read wal
list of archived wal files
If last_wal is provided, the method will discard all the wall files older than last_wal. If last_position is provided the method will try to read the xlog.db file using last_position as starting point. If the wal file at last_position does not match last_wal, read from the start and use last_wal as limit
- sync_wals()View on GitHub#
Method for the synchronisation of WAL files on the passive node, by copying them from the primary server.
The method checks if the server is passive, then tries to acquire a sync-wal lock.
Recovers the id of the last locally archived WAL file from the status file ($wals_directory/sync-wals.info).
Reads the primary.info file and parses it, then obtains the list of WAL files that have not yet been synchronised with the master. Rsync is used for file synchronisation with the primary server.
Once the copy is finished, acquires a lock on xlog.db, updates it then releases the lock.
Before exiting, the method updates the last_wal and last_position fields in the sync-wals.info file.
- Parameters:
self (barman.server.Server) – the Server object to synchronise
- property systemidView on GitHub#
Get the system identifier, as returned by the PostgreSQL server :return str: the system identifier
- wait_for_wal(wal_file=None, archive_timeout=None)View on GitHub#
Wait for a WAL file to be archived on the server
- write_identity_file()View on GitHub#
Store the identity of the server if it doesn’t already exist.
- write_sync_wals_info_file(primary_info)View on GitHub#
Write the content of SYNC_WALS_INFO_FILE on disk
- Parameters:
primary_info (dict)
- xlogdb(mode='r')View on GitHub#
Context manager to access the xlogdb file.
This method uses locking to make sure only one process is accessing the database at a time. The database file will be created if it not exists.
Usage example:
- with server.xlogdb(‘w’) as file:
file.write(new_line)
- Parameters:
mode (str) – open the file with the required mode (default read-only)
- property xlogdb_file_nameView on GitHub#
The name of the file containing the XLOG_DB :return str: the name of the file that contains the XLOG_DB
- class barman.server.SyncWalInfo(last_wal, last_position)#
Bases:
tuple
- _asdict()#
Return a new dict which maps field names to their values.
- _field_defaults = {}#
- _fields = ('last_wal', 'last_position')#
- classmethod _make(iterable)#
Make a new SyncWalInfo object from a sequence or iterable
- _replace(**kwds)#
Return a new SyncWalInfo object replacing specified fields with new values
- last_position#
Alias for field number 1
- last_wal#
Alias for field number 0