aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--NOTES140
-rw-r--r--README66
-rw-r--r--TODO29
-rw-r--r--destination169
-rw-r--r--files153
-rw-r--r--local123
-rw-r--r--remote92
-rwxr-xr-xrsync-backup648
-rw-r--r--source69
-rw-r--r--variables211
-rw-r--r--write68
11 files changed, 1125 insertions, 643 deletions
diff --git a/NOTES b/NOTES
new file mode 100644
index 0000000..cc84920
--- /dev/null
+++ b/NOTES
@@ -0,0 +1,140 @@
+This is intended as a short, rather high-level, documentation.
+
+
+# -- NOTATIONS -- #
+
+ Define a few notations, as used in the following.
+
+- file tree: A file tree with possible excluded subtrees residing on exactly
+ one file system. The file tree does not need to share the root
+ of that file system.
+
+
+
+# -- GENERAL -- #
+
+ This backup program makes incremental backups of a file tree, using hard
+links between differently dated backups of that file tree to reduce space
+requirements while keeping the data in a simple form.
+
+ The destination file system is usually different to the on the source file
+tree resides on, in particular, it may be on a different host. The rsync(1)
+command line program serves as backend. Therefore, at least one of the
+source and destination need to be local.
+
+ The backup process is organized in a way such that any pure file renames
+between the old and new backup directory do not cause a new file to be
+created. I.e., the two files are hard linked. This is contrary to the
+simpler method of simply exploiting rsync's '--link-dest' functionality, and
+thereby to the early versions of this program.
+
+ This is achieved by
+
+ 1. Creating a local mirror of the source tree inside itself
+ (non-recursively, of course), using hard links. In common backup
+ operation, there are always at least two such mirrors: an obsolete one,
+ a remnant of the last backup's creation, featuring the same structure
+ as the last backup as stored in the destination file tree, and the newly
+ created mirror. By transitivity, the two mirrors share hard links. See
+ also APPENDIX: NOTE-1.
+
+ 2. Copying the pair of these two mirrors to the destination tree, thereby
+ referencing the old mirror to the (identical!) old mirror in the
+ destination tree. Since the two mirrors are inter-linked, and the
+ referencing allows for linking the old mirror to the (old) latest backup,
+ the new mirror (and very latest backup) is effectively linked to the
+ old latest backup. See also APPENDIX: NOTE-2.
+
+
+ The chosen procedure has some side effects.
+
+ * Backups are available on the source host (however they likely diverge from
+ the actual backups, as the former are linked to the live file tree and
+ hence change upon in-place modification). See also APPENDIX: NOTE-1.
+ `- Note that, currently, neither in the source nor in the destination tree,
+ backups (resp. mirrors) are deleted. In the source tree this might be
+ more desired than in the destination tree.
+
+ * The local mirror creation is much faster than the transfer to the backup
+ medium, in particular, if it is remote. Therefore, the state, excluding
+ the aforementioned in-place modifications, is closer to consistency with a
+ single point in time. One might argue though, that precisely these
+ in-place modifications cause the window of modifications to be in fact
+ widened to the adjunction of the mirroring and the transfer.
+
+
+ There are also some clear (but probably not grave) drawbacks to the simpler
+method.
+
+ * There is one more step involved, thereby adding complexity.
+
+ * Both the source and destination file tree need to reside on respectively
+ one single file system. As a consequence, this program needs to be
+ configured for each file system containing data to be backed up. This
+ could in theory be done in the program itself, but seems not worth the
+ effort.
+
+ * A little additional space is needed on the source file system, however
+ only for new file links and directories, not for new actual regular files.
+ `- In particular though, the source file system needs to be writable. This
+ problem may possibly be circumvented using bind mounts.
+
+
+
+# -- REMOTE METHOD CALLS -- #
+
+ This program essentially wraps rsync, however performs a few additional,
+useful functions, some of which need to be executed on the source or
+destination host, irrespective of whether that is local or not. In order to
+allow for this, there exist wrapper methods, namely 'run_function' and
+'run_command', allowing to execute a local bash function on a remote host.
+Methods to be executed either on the source or the destination are located in
+the source files 'source' and 'destination' and annotated with environment
+dependencies ('requires'). See also the 'remote' source file.
+
+
+
+# -- APPENDIX -- #
+
+# NOTE-1: The old mirror.
+ As the old mirror is a remnant of the last backup it features exactly the
+ same file tree as that last backup - by labels, not by content. This is
+ due to this old mirror being (purposefully) linked to the live file tree,
+ in which files may be modified in place, which due to the hard links also
+ appear in the old mirror.
+
+
+# NOTE-2: Avoid propagation of errors on the old source mirror.
+ The simple way to do the transfer from the source to the destination tree
+ may seem to use the old latest backup on the destination as part of the
+ target, when referencing the old mirror to it.
+
+ I.e., sync (old_mirror, new_mirror) -> (old_latest_backup, new_backup).
+ This is however, not a good idea, since as explicated in NOTE-1, old_mirror
+ may change and hence the above operation would change old_latest_backup.
+ Since this is not desired, old_latest_backup is instead only used as
+ link-dest for rsync, which unfortunately causes a superfluous instance of
+ old_latest_backup to be created in the destination tree. This is simply
+ deleted in the end.
+
+
+# NOTE-3: Imperfection.
+ The chosen solution is far from perfect. In particular, the backups are
+ far from being atomar. Apparently, the Btrfs makes this easier [0].
+ Another alternative might be to use an overlayFS to record new writes.
+
+ Further, in order to at least get closer to atomicity, a full copy, made
+ on and of the source tree, could be done before actual transferral. This
+ would however require a lot of space and also only likely be beneficial, if
+ the transfer is non-locally.
+
+ Also, the extranously created copy of the source's old mirror on the
+ destination is disturbing. I do however not know of way to avoid this,
+ unless not relying on the rsync(1) command line program as backend.
+
+
+# Links
+[0] https://btrfs.wiki.kernel.org/index.php/Incremental_Backup
+
+
+vi: ai et tw=77 fo+=t
diff --git a/README b/README
index 1287b72..66ba148 100644
--- a/README
+++ b/README
@@ -1,23 +1,77 @@
-- in work -
+Introduction
+------------
+
+ rsync-backup is a frontend to rsync (with a few extras) to allow for the
+creation of incremental backups, using hard links. It notably is tracking
+movements, that is a file moved (but unchanged) in the source file tree will
+not be stored twice for two corresponding backups - before and after the
+movement.
Dependencies
------------
-Most dependencies are required locally and remotely, as a huge part of the
+ Most dependencies are required locally and remotely, as a huge part of the
script may be executed remotely.
- bash>=4.2 (test -v)
- rsync
- ssh
-- coreutils (realpath, date)
+- coreutils
- find
+Installation
+------------
+
+ Put all the source files in one directory. It may be desirable to set up
+a symbolic link in e.g. /usr/local/bin, or a shell alias.
+
+
+Usage
+-----
+
+ Execute 'rsync-backup' either with one argument, specifying the source
+directory, requiring that to be configured (see Configuration) or two
+arguments, where the second one specifies the destination.
+
+ It is highly recommended to configure the respective directories, not least
+to configure to be excluded parts of the file tree (e.g., /dev, /mnt).
+
+ It is possible to use a remote location for one of the source and
+destination.
+
+ Examples:
+$ rsync-backup /home # requires prior configuration
+$ rsync-backup / user@server:/backup
+
+
+Configuration
+-------------
+
+ Configuration may be done in the file '/etc/rsync-backup/config'. See the
+source file 'variables' for information on which variables can be set.
+
+
+Documentation
+-------------
+
+As of now, documentation on the usage is found only here.
+
+The file NOTES provides an overview of the ideas and the (inner) working of
+this program.
+
+
Notes
-----
-Symbolic links are simply copied, thus linking to the backed-up filetree, not
-the backup-filetree itself.
+ IMPORTANT: Both the source and destination file tree should reside on a
+single file system. It is highly recommended to exclude (config:
+filter_file) other mounted file systems. Otherwise, assuming the backup
+process does not fail, each new backup is expected to take the full space for
+such mounted subdirectories.
+
+ Symbolic links are simply copied, thus linking to the backed-up filetree,
+not the backup-filetree itself.
-Dates are always calculated on the local host.
+ Dates are always calculated on the source host.
diff --git a/TODO b/TODO
index caa934b..142a7bb 100644
--- a/TODO
+++ b/TODO
@@ -1,4 +1,8 @@
# -- BUGS -- #
+- /path/ is not recognized as configured, when only /path explicitly is
+ configured.
+- After a failure, the .backup directory is in a bad state (contains backup
+ data labeled as old, however not related to the destination).
# -- TODO -- #
- For ssh, set up one single connection (maybe configurable?).
@@ -6,13 +10,11 @@
-oControlMaster=yes \
-oControlPersist $remote /bin/true
`- ssh -oControlPath=/tmp/root-readable/something.sock
-- Replace echo by printf(-functions).
- Set up default configuration file.
- [consider] When installing, copy the config file to both /etc and
/usr/share, such that sourcing from the latter location removes the need to
specify default options in the script itself.
- Test for read/write access at some point.
-- Better error reporting.
- Use autotools.
- Per dirpath rsync_args.
- Allow for resuming of backups.
@@ -21,7 +23,7 @@
`- Not a good idea if a notable amount of time passed.
`- [question] What happens in case of partially transferred files?
`- --delete-during, --delete-excluded
-- Shell completion of configured backup paths.
+- [consider] Shell completion of configured backup paths.
- Better error handling on failure (of rsync).
- Use ask().
`- Allow for configuration of default answer.
@@ -29,6 +31,12 @@
- [consider] Get date on source host.
- Named backups to allow for different destinations per source (and shorter
names).
+- Test suite.
+- Check for several file systems in a tree (and error upon locating such).
+- [consider] Add fallback if no local mirror of the source file tree is
+ available.
+- Only store the last mirror locally, but date it.
+- [consider] --one-file-system as default option.
# -- PROBLEMS -- #
- rsync bug: https://bugzilla.samba.org/show_bug.cgi?id=13445
@@ -56,3 +64,18 @@
`- For now, -v or -vv in rsync_args should work fine.
- non-intercative option.
- rsync_network_options (when remote src or dest, e.g. for --compress).
+- Write metadata.
+- Use wrappers for the functions possibly non locally executed.
+- Compare src:work/$old_date and dest:$old_date for consistency.
+- Run script instead of function remotely
+ `- ssh host 'bash -s' -- < script_file args
+ `- In case of dependencies, the otherwise sourced files could just be
+ included in the stdin of ssh.
+ `- cat script_file dep_file | ssh host 'bash -s' -- args
+- Use stdout of functions to return a string value and stderr for everything
+ else.
+ `- Optional: Prefixes determining whether normal or error.
+ `- Convert at the end (remove prefix and pipe to stderr / stdout).
+- Only keep the respectively latest backup on the source.
+ `- Or the last two, which would allow to reuse the second-last as new
+ mirror (for a probably negligible gain, if any).
diff --git a/destination b/destination
new file mode 100644
index 0000000..4829838
--- /dev/null
+++ b/destination
@@ -0,0 +1,169 @@
+#!/usr/bin/env bash
+#
+# destination - functions acting on the destination host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Setup working directory on the destination files system.
+#
+# requires: create_new_dir
+#
+function dest_prepare_initial
+{
+ create_new_dir "$dest_path" || return $?
+ create_new_dir "${dest_path}/work" || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Setup working directory on the destination file system, check for too new
+# (future) backups and refresh the 'latest' symlink.
+#
+# requires: assert_is_dir, dest_list_backup_dirs, create_new_dir
+# $dest_path
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function dest_prepare_incremental
+{
+ local work_dir="${dest_path}/work"
+ local old_date="$1"
+ local new_date="$2"
+ local latest
+
+ assert_is_dir "$dest_path" || return $?
+
+ latest="$(dest_list_backup_dirs | tail -1)"
+
+ if [[ "$old_date" != "$latest" ]]
+ then
+ error "Dates of last backup do not match on source and destination."
+ debug_error "source: ${old_date} | dest: ${latest}"
+ return $RET_FILE_ERR
+ fi
+
+ if ! [[ "$latest" < "$new_date" ]]
+ then
+ error "Latest backup on destination not older than source."
+ return $RET_FILE_ERR
+ fi
+
+ # Refresh 'latest' symlink in case it was obsoleted or corrupted.
+ ln -sfT "$latest" "${dest_path}/latest" || return $RET_ERROR
+
+ create_new_dir "$work_dir" || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Finalize the backup creation on the destination host.
+#
+# requires: dest_refresh_symlinks
+# `-> assert_is_dir, dest_list_backup_dirs,
+# replace_symlink_target, create_dir
+# $dest_path (self, <- dest_refresh_symlinks),
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function dest_finalize
+{
+ local old_date="$1"
+ local new_date="$2"
+
+ message "Finish installment of new backup..."
+ mv -T "${dest_path}/work/${new_date}" "${dest_path}/${new_date}" \
+ || return $RET_ERROR
+
+ message "Remove auxiliary directory tree..."
+ rm -rf "${dest_path}/work/${old_date}" || return $RET_ERROR
+ rmdir "${dest_path}/work" || return $RET_ERROR
+
+ dest_refresh_symlinks || return $?
+
+ message 'Successfully finished installment of new backup.'
+}
+
+
+# Print existing backup directories.
+# Must be run on destination host.
+#
+# requires: $dest_path
+#
+function dest_list_backup_dirs
+{
+ find "$dest_path" -mindepth 1 -maxdepth 1 -type d \
+ -regextype posix-extended -regex \
+ '.*/[0-9]{4}-[0-1][0-9]-[0-3][0-9]_[0-2][0-9]([0-5][0-9]){2}' \
+ -printf '%f\n' \
+ | LC_ALL=C sort
+
+ if [[ ${PIPESTATUS[0]} -eq 0 && ${PIPESTATUS[1]} -eq 0 ]]
+ then
+ return $RET_SUCCESS
+ else
+ return $RET_ERROR
+ fi
+}
+
+
+# Create symbolic links to backup directories.
+#
+# requires: assert_is_dir, dest_list_backup_dirs, replace_symlink_target,
+# create_dir
+# $dest_path
+#
+function dest_refresh_symlinks
+{
+ local -i i
+ local -i n
+ local -a backup_dirs
+ local link
+ local ret
+
+ assert_is_dir "$dest_path" || return $?
+
+ backup_dirs=( $(dest_list_backup_dirs || return $?) )
+ n=${#backup_dirs[@]}
+
+ replace_symlink_target "${dest_path}/latest" "${backup_dirs[n-1]}" \
+ || return $?
+
+ ## by_number
+ numdir="${dest_path}/by_number"
+ create_dir "$numdir" || return $?
+
+ # Remove old symlinks.
+ find "$numdir" -mindepth 1 -maxdepth 1 -type l -regextype posix-extended \
+ -regex '.*/[0-9]*' -delete || return $RET_ERROR
+
+ for (( i = 0; i < n; i++ ))
+ do
+ ln -s "../${backup_dirs[i]}" "$(printf "%s/%.${#n}u" "$numdir" $i)" \
+ || return $RET_FILE_ERR
+ done
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/files b/files
new file mode 100644
index 0000000..85640e7
--- /dev/null
+++ b/files
@@ -0,0 +1,153 @@
+#!/usr/bin/env bash
+#
+# files - functions acting on files.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Create directory, unless it exists already. Returns non-zero, if the
+# named file exists however is not a directory.
+#
+# $1: directory: path
+#
+function create_dir
+{
+ local dir="$1"
+
+ if [[ ! -e "$dir" && ! -L "$dir" ]]
+ then
+ mkdir "$dir" || return $RET_ERROR
+ elif [[ ! -d "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" is not a directory."
+ return $RET_FILE_ERR
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Create a new directory, assuring it did not exist before.
+#
+# $1: directory: path
+#
+function create_new_dir
+{
+ local dir="$1"
+
+ if [[ -e "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" exists already."
+ return $RET_FILE_ERR
+ fi
+
+ mkdir "$dir" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Verify the specified file is a directory.
+#
+# $1: directory: path
+#
+function assert_is_dir
+{
+ local dir="$1"
+
+ if [[ ! -d "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" is not a directory."
+ return $RET_FILE_ERR
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Replace symbolic link $2 by symbolic link $1. $2 may not exist.
+#
+# $1: source file name: path
+# $2: target file name: path
+#
+function replace_symlink
+{
+ local src="$1"
+ local tgt="$2"
+
+ if [[ ! -L "$src" ]]
+ then
+ error "\"$src\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ if [[ ! -L "$tgt" && -e "$tgt" ]]
+ then
+ error "\"$tgt\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ mv -T "$src" "$tgt" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Replace symbolic link $1's target by $2. $1 may not exist.
+#
+# $1: link file: path
+# $2: link target: path
+#
+function replace_symlink_target
+{
+ local link="$1"
+ local tgt="$2"
+
+ if [[ ! -L "$link" && -e "$link" ]]
+ then
+ error "\"$link\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ ln -sfT "$tgt" "$link" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Remove a symbolic link.
+#
+# $1: link: path
+#
+function remove_symlink
+{
+ local file="$1"
+
+ if [[ ! -L "$file" ]]
+ then
+ error "\"$file\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ rm "$file" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/local b/local
new file mode 100644
index 0000000..37b2574
--- /dev/null
+++ b/local
@@ -0,0 +1,123 @@
+#!/usr/bin/env bash
+#
+# local - functions acting on the local host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+function main
+{
+ local src_old_date
+ local src_new_date
+ local src_mirrors_path
+
+ shopt -s $SHOPTS
+
+ # Set $src, $dest_path and $filter_args.
+ get_args "$@" || return $?
+
+ shopt -s $SHOPTS_AFTER_GET_ARGS
+
+ run_function "$src_host" src_prepare \
+ 'assert_is_dir create_dir replace_symlink' \
+ 'src_path src_mirror_reldir RSYNC rsync_args[@] filter_args[@] DATEFMT' \
+ || return $?
+
+ src_mirrors_path="${src_path}/${src_mirror_reldir}"
+ src_old_date="$(
+ run_command "$src_host" readlink "${src_mirrors_path}/old")"
+ src_new_date="$(
+ run_command "$src_host" readlink "${src_mirrors_path}/new")"
+
+ if [[ "$src_old_date" == 'none' ]]
+ then
+ run_function "$dest_host" dest_prepare_initial 'create_new_dir' '' \
+ || return $?
+
+ transfer_initial_backup "$src_new_date" || return $?
+ else
+ run_function "$dest_host" dest_prepare_incremental \
+ 'assert_is_dir dest_list_backup_dirs create_new_dir' 'dest_path' \
+ "$src_old_date" "$src_new_date" \
+ || return $?
+
+ transfer_incremental_backup "$src_old_date" "$src_new_date" || return $?
+ fi
+
+ run_function "$dest_host" dest_finalize \
+ 'dest_refresh_symlinks
+ assert_is_dir dest_list_backup_dirs replace_symlink_target
+ create_dir' \
+ 'dest_path' \
+ "$src_old_date" "$src_new_date" \
+ || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Transfer the very first backup from source to destination. To be run after
+# src_prepare and dest_prepare.
+#
+# $1: date of the backup: date in $DATEFMT
+#
+function transfer_initial_backup
+{
+ local date="$1"
+
+ message "Transfer new (initial) backup..."
+
+ "$RSYNC" "${rsync_args[@]}" \
+ "${src}/${src_mirror_reldir}/${date}" \
+ "${dest}/work" \
+ || return $RET_ERROR
+
+ message "Finished transfer."
+
+ return $RET_SUCCESS
+}
+
+
+# Transfer new backup from source to destination. To be run after
+# src_prepare and dest_prepare.
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function transfer_incremental_backup
+{
+ local old_date="$1"
+ local new_date="$2"
+
+ message "Transfer new (incremental) backup..."
+
+ # Note: Use --link-dest instead of making the old backup part of the target
+ # directory, since src/old_date may have changed due to in-place
+ # modifications.
+ "$RSYNC" "${rsync_args[@]}" --link-dest='..' \
+ "${src}/${src_mirror_reldir}"/{"$old_date","$new_date"} \
+ "${dest}/work" \
+ || return $RET_ERROR
+
+ message "Finished transfer."
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/remote b/remote
new file mode 100644
index 0000000..1b04197
--- /dev/null
+++ b/remote
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+#
+# remote - execute functions remotely.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Run command remotely.
+#
+# $1: host: name
+# ${@:2}: command with arguments: string+
+#
+function run_command
+{
+ local host="$1"
+ shift
+
+ if [[ -z "$host" ]]
+ then
+ "$@"
+ else
+ # Use ssh to run command remotely.
+ # Make sure to properly escape the strings.
+ ssh "$host" "shopt -s ${SHOPTS}; $(printf "'%s' " "$@")"
+ fi
+
+ return $?
+}
+
+
+# Run a function on a remote host.
+# If the host parameter is empty, run locally.
+#
+# $1: host: name
+# $2: function to run: name
+# $3: other functions to export: [name (. ' ' . name)*]
+# $4: variables to export: [name (. '[@]')? (. ' ' . name (. '[@]')?)*]
+# ${@:5}: function arguments: string+
+#
+function run_function
+{
+ local host="$1"
+ local fun=$2
+ local funs="$fun $3 $REMOTE_DEFAULT_FUNS"
+ local vars="$4 $REMOTE_DEFAULT_VARS"
+ local vars_set
+
+ shift 4
+
+ if [[ -z "$host" ]]
+ then
+ $fun "$@"
+ else
+ # Create string to export variables.
+ vars_set=
+
+ # Avoid surprises of Bash's globbing (due to brackets in, e.g., a[@]).
+ set -o noglob
+
+ for var in $vars
+ do
+ vars_set+="$(eval echo "\${${var}@A}"); "
+ done
+
+ set +o noglob
+
+ # Use ssh to run command remotely.
+ # Make sure to properly escape the strings.
+ ssh "$host" "shopt -s ${SHOPTS}; ${vars_set} $(typeset -f $funs);" \
+ "$fun $(printf "'%s' " "$@")"
+ fi
+
+ return $?
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/rsync-backup b/rsync-backup
index 0a7df33..86bff28 100755
--- a/rsync-backup
+++ b/rsync-backup
@@ -21,642 +21,22 @@
#
-typeset -r exec_name="${0##*/}"
+typeset -r full_exec_name="$(realpath "$0")"
+typeset -r exec_name="${full_exec_name##*/}"
+typeset -r source_code_dir="${full_exec_name%/*}"
+cd "$source_code_dir"
-# CONSTANTS
-typeset -r CONFIG=/etc/rsync-backup/config
-typeset -r RET_TYPES='
- RET_SUCCESS RET_FAILURE RET_BADSYNTAX RET_FILE_ERR RET_ERROR'
-typeset -r RET_SUCCESS=0
-typeset -r RET_FAILURE=1
-typeset -r RET_BADSYNTAX=2
-typeset -r RET_FILE_ERR=4
-typeset -r RET_ERROR=8
-typeset -r DATEFMT='%Y-%m-%d_%H%M%S'
-
-# Shell options:
-# nullglob: In case of a non-matching glob, return nothing.
-# dotglob: Include dotfiles (except . and ..) in glob matching.
-typeset -r SHOPTS='dotglob'
-typeset -r SHOPTS_AFTER_GET_ARGS='nullglob'
-
-
-# Global program variables.
-typeset src src_path src_host
-typeset dest dest_path dest_host
-typeset -a filter_args
-typeset -i n
-typeset bakdate
-
-
-# Variables that may be configured.
-typeset RSYNC=rsync
-typeset -a rsync_args
-typeset -a rsync_inc_args
-typeset -A bakpath
-typeset filter_file_all # Applied after specific filter.
-typeset -A filter_file
-
-# Default arguments for rsync (-rptogAXlHS, --timeout=, --info=).
-rsync_args=(
- --recursive
- --perms --times --owner --group
- --acls --xattrs
- --links
- --hard-links
- --sparse
- --timeout=60
- #-vv
- --info=progress2
- )
-
-# Supplemental arguments for rsync; only used when an actual incremental
-# update is performed, i.e. when --link-dest is used.
-rsync_inc_args=(
- --fuzzy --fuzzy
- )
-
-filter_file_all=/etc/rsync-backup/filter
-
-source "$CONFIG"
-
-
-# RSYNC args.
-# basic args: --recursive --perms --times --owner --group --links
-# extra args: --sparse --acls --xattrs --hard-links --timeout=60
-# --fuzzy --fuzzy
-# --filter="merge <file>"
-# likely: --compress
-# --partial{,dir=DIR}
-# --progress
-# possibly: --devices --specials
-# --max-size=SIZE-OF-FILE
-# --one-file-system
-# --log-file=FILE --log-file-format=FORMAT
-# --human-readable
-# unlikely: --omit-{dir,link}-times --update (!--inplace) --delete
-# --exclude
-# great: --link-dest=DIR (timestamps?)
-# testing: --verbose --dry-run (-vn)
-# interesting: --sockopts --itemize-changes --out-format=FORMAT
-# --stats
-
-function main
-{
- shopt -s $SHOPTS
-
- # Set $src, $dest_path and $filter_args.
- get_args "$@" || return $?
-
- shopt -s $SHOPTS_AFTER_GET_ARGS
-
- # Verify $src_path exists.
- run_function "$src_host" src_check_path '' 'src_path src' || return $?
-
- # Make sure $dest_path exists.
- run_function "$dest_host" dest_check_path '' 'dest_path dest' || return $?
-
- # Populate the global $dirs array.
- dirs=( $(
- run_function "$dest_host" dest_find_existing_backup_dirs '' 'dest_path' \
- || return $?
- ) )
-
- n=${#dirs[@]}
-
- run_function "$dest_host" dest_prepare 'dest_clear_file' \
- 'dest dirs[@] n dest_host dest_path' "$(date +$DATEFMT)" \
- || return $?
-
- perform_backup || return $?
-
- run_function "$dest_host" dest_finalize \
- 'dest_refresh_symlinks dest_clear_file' \
- 'dest_host dest_path bakdate n' \
- || return $?
-
- dest_refresh_symlinks
-
- return $RET_SUCCESS
-}
-
-
-# Set the globabl variables $src, $dest_path and $filter_args according to
-# command line parameters and the configuration.
-#
-# $@: command line parameters
-#
-# requires the shell option nullglob to be _unset_. (for test -v to work)
-#
-function get_args
-{
- if [ $# -eq 0 ]
- then
- echo "Usage: $exec_name <src> [<dest>]"
- return $RET_BADSYNTAX
- fi
-
- local l_src="$1"
- local l_dest
-
- # Set $src_path, $src_host and $src.
- resolve_host_path src "$l_src" || return $?
-
- if [ $# -gt 1 ]
- then
- l_dest="$2"
- elif test -v bakpath[$src] # $# -eq 1
- then
- l_dest="${bakpath[$src]}"
- else
- echo "Error: Backup path for $src neither configured"
- echo " nor specified on command line."
-
- return $RET_BADSYNTAX
- fi
-
- # Set $dest_path, $dest_host and $dest.
- resolve_host_path dest "$l_dest" || return $?
-
- filter_args=()
- if test -v filter_file[$src] && test -f "${filter_file[$src]}"
- then
- filter_args+=("--filter=merge ${filter_file[$src]}")
- fi
- if test -v filter_file_all -a -f "$filter_file_all"
- then
- filter_args+=("--filter=merge ${filter_file_all}")
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Split host:target like rsync and convert local (!) relative paths to
-# absolute ones. This is needed at least for $src_path which is used as
-# key for associative arrays.
-#
-# The results are written to the global variables $<vtype>, $<vtype>_host
-# and $<vtype>_path, where <vtype> is either src or dest. $<vtype> gets
-# set to the original to be resolved string, unless we are local, i.e.
-# there is no host.
-#
-# Note: This is not a place for bash to shine. Perl for instance would do
-# this much more easily.
-#
-# Examples:
-# - a:b -> (a, b)
-# - a/b:c -> ('', a/b:c)
-#
-# $1: which global variables to set: (src|dest)
-# $2: path string potentially containing host prefix
-#
-# sets: ${$1}_host: host name (may be empty)
-# ${$1}_path: path (absolute, if local, i.e. ${$1}_host='')
-# ${$1}: ${$1}_path, if ${$1}_host='',
-# ${$1}_host:${$1}_path (= $2), elsewise.
-#
-function resolve_host_path
-{
- local vtype="$1"
- local combined="$2"
- local host path
-
- # Split host and path.
- host=${combined%%:*}
- if [[ "$host" == "$combined" || "$host" =~ '/' ]]
- then
- host=''
- path="$combined"
-
- # Resolve to absolute directory.
- # pwd -P avoids symlinks.
- # Do not resolve symlimks in ${path} or test for its existance.
- # `- realpath could do that.
- if ! [[ "$path" =~ / ]]
- then
- path="$(pwd -P)/${path}"
- fi
-
- combined="$path"
- else
- path="${combined#*:}"
- fi
-
- # Set global variables.
- if [[ "$vtype" == 'src' ]]
- then
- src_host="$host"
- src_path="$path"
- src="$combined"
- else # "$vtype" == 'dest'
- dest_host="$host"
- dest_path="$path"
- dest="$combined"
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Run a function on a remote host.
-# If the host parameter is empty, run locally.
-#
-# $1: host: name
-# $2: function to run: name
-# $3: other functions to export: [name (. ' ' . name)*]
-# $4: variables to export: [name (. '[@]')? (. ' ' . name (. '[@]')?)*]
-# ${@:5}: function arguments: string+
-#
-function run_function
-{
- local host="$1"
- local fun=$2
- local funs="$fun $3"
- local vars="$4"
- local vars_set
-
- shift 4
-
- if [[ -z "$host" ]]
- then
- $fun "$@"
- else
- # Create string to export variables.
- vars_set=
- for var in $RET_TYPES $vars
- do
- vars_set+="$(eval echo \${${var}@A}); "
- done
-
- # Use ssh to run command remotely.
- # Make sure to properly escape the strings.
- ssh "$host" "shopt -s ${SHOPTS}; ${vars_set} $(typeset -f $funs);" \
- "$fun $(printf "'%s' " "$@")"
- fi
-
- return $?
-}
-
-
-# Run command remotely.
-#
-# Not used.
-#
-# $1: host: name
-# ${@:2}: command with arguments: string+
-#
-function do_thing
-{
- local host="$1"
- shift
-
- if [[ -z "$host" ]]
- then
- "$@"
- else
- # Use ssh to run command remotely.
- # Make sure to properly escape the strings.
- ssh "$host" "shopt -s ${SHOPTS}; $(printf "'%s' " "$@")"
- fi
-
- return $?
-}
-
-
-# Check if $src exists.
-# Must be run on source host.
-#
-# requires: $src_path, $src
-#
-function src_check_path
-{
- if ! test -d "$src_path"
- then
- echo "Source (\"$src\") does not exist."
- return $RET_FILE_ERR
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Check if $dest_path exists and try to create it if necessary.
-# Must be run on destination host.
-#
-# requires: $dest_path, $dest
-# interactive.
-#
-function dest_check_path
-{
- local -l reply
-
- if ! test -d "$dest_path"
- then
- echo "Target (\"$dest\") does not exist."
- echo -n "Do you want do create it [y/N]? "
- read reply
- if ! [[ "$reply" =~ ^y(es)?$ ]]
- then
- return $RET_FAILURE
- fi
- echo
-
- if ! mkdir -p "$dest_path"
- then
- echo "Backup creation has failed."
- echo "\"$dest\" could not be created."
- return $RET_FILE_ERR
- fi
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Print existing backup directories.
-# Must be run on destination host.
-#
-# requires: $dest_path
-#
-function dest_find_existing_backup_dirs
-{
- find "$dest_path" -mindepth 1 -maxdepth 1 -type d \
- -regextype posix-extended -regex \
- '.*/[0-9]{4}-[0-1][0-9]-[0-3][0-9]_[0-2][0-9]([0-5][0-9]){2}.backup' \
- -printf '%f\n' \
- | LC_ALL=C sort
-
- if [[ ${PIPESTATUS[0]} -eq 0 && ${PIPESTATUS[1]} -eq 0 ]]
- then
- return $RET_SUCCESS
- else
- return $RET_ERROR
- fi
-}
-
-
-# Prepare destination for the backup process.
-# Must be run on destination host.
-#
-# requires: $dirs[@], $n
-# $dest_host (self, <- dest_clear_file)
-# $dest_path
-#
-# $1: current date on local host in format $DATEFMT
-#
-# interactive (<- dest_clear_file)
-#
-function dest_prepare
-{
- local current_date="$1"
-
- # Check for dates in the future, which might mess with the yet to be
- # determined new backup's date.
- if [[ $n -gt 0 ]]
- then
- local last_date=${dirs[n-1]%.backup}
-
- if [[ "$current_date" == "$last_date" ]]
- then
- echo "Last backup was run in the same second."
- echo "Exiting."
- return $RET_FAILURE
- elif
- [[ "$current_date" < "$last_date" ]]
- then
- echo "Last backup lies in the future. Fix your clocks."
- echo "Exiting."
- return $RET_FAILURE
- fi
- fi
-
- dest_clear_file "${dest_path}/new.backup" "Backup creation has failed." \
- || return $?
-
- return $RET_SUCCESS
-}
-
-
-# Perform the actual backup.
-# To be run locally.
-#
-# requires: $DATEFMT,
-# $dest_path, $dest, $src_path, $src,
-# $n, $dirs[@],
-# $rsync_args[@], $rsync_inc_args[@], $filter_args[@]
-#
-# sets: $bakdate
-#
-function perform_backup
-{
- bakdate=`date +"$DATEFMT"`
-
- printf 'Starting backup nr. %u.\n' "$n"
- if [ $n -eq 0 ]
- then
- # Perform first backup, not incremental.
- $RSYNC "${rsync_args[@]}" \
- "${filter_args[@]}" \
- "$src"/ "${dest}/new.backup" \
- || return $RET_ERROR
- else
- # Perform incremental backup on the basis of the last.
- $RSYNC "${rsync_args[@]}" \
- "${rsync_inc_args[@]}" --link-dest="../${dirs[n-1]}" \
- "${filter_args[@]}" \
- "$src"/ "${dest}/new.backup" \
- || return $RET_ERROR
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Finalize the backup creation on the destination host.
-#
-# requires: $dest_host (<- dest_refresh_symlinks),
-# $dest_path (self, <- dest_refresh_symlinks),
-# $bakdate (self, <- dest_refresh_symlinks),
-# $n (self, <- dest_refresh_symlinks),
-#
-# interactive (<- dest_refresh_symlinks)
-#
-function dest_finalize
-{
- mv "${dest_path}/new.backup" "${dest_path}/${bakdate}.backup" \
- || return $RET_FILE_ERR
-
- if
- dest_refresh_symlinks
- then
- printf '\nCreating of backup nr. %u successfully executed.\n' "$n"
- return $RET_SUCCESS
- else
- local ret=$?
- printf '\nCreating of backup nonetheless successfully executed.\n'
- return $ret
- fi
-}
-
-
-# Create symbolic links to backup directories.
-# Must be run on the destination host.
-#
-# requires: $dest_host, $dest_path, $bakdate, $n
-# interactive (<- dest_clear_file)
-#
-function dest_refresh_symlinks
-{
- local -i i
- local host_prefix="${dest_host}${dest_host:+:}"
- local linkdir ret
-
- linkdir="${dest_path}/latest"
- if test -L "$linkdir"
- then
- rm "$linkdir"
- ln -s "${bakdate}.backup" "$linkdir" || return $RET_ERROR
- elif
- dest_clear_file "$linkdir" "Creation of softlink \"latest\" failed." \
- || return $?
- then
- ln -s "${bakdate}.backup" "$linkdir" || return $RET_ERROR
- fi
-
- # by_number
- numdir="${dest_path}/by_number"
- if ! test -d "$numdir"
- then
- dest_clear_file "$dest_host" "$numdir" \
- "Setting up numbered softlinks failed." || return $?
-
- mkdir "$numdir" || return $RET_ERROR
- fi
-
- # Remove old symlinks.
- find "$numdir" -mindepth 1 -maxdepth 1 -type l -regextype posix-extended \
- -regex '.*/[0-9]*' -delete || return $RET_ERROR
-
- # TODO: Simplify.
- for file in "$numdir"/*
- do
- echo $file
- dest_clear_file "$file" "Setting up numbered softlinks failed." \
- || return $?
- done
-
- # Set last dir on the new one.
- dirs[$n]="${bakdate}.backup"
-
- for (( i = 0; i <= n; i++ ))
- do
- ln -s "../${dirs[i]}" "$(printf "%s/%.${#n}u" "$numdir" $i)"
- done
-
- return $RET_SUCCESS
-}
-
-
-# Make sure a file does not exist on the destination host.
-# Must be run on destination host.
-#
-# $1: file: path
-# $2: error message: string
-#
-# requires: $dest_host
-# interactive.
-#
-function dest_clear_file
-{
- local file_path="$1"
- local err_msg="$2"
- local file="${dest_host}${dest_host:+:}${file_path}"
-
- local -l reply
-
- if test -e "$file_path" -o -L "$file_path"
- then
- printf "%s %s\n %s\n%s\n" \
- "It seems like the last backup process failed or you have created" \
- "the file" "$file" "manually."
-
- echo -n "Do you want to delete it [y/N]? "
- read reply
-
- if [[ "$reply" =~ ^y(es)?$ ]]
- then
- printf "Deleting.\n"
-
- if rm -rf "$file_path"
- then
- return $RET_SUCCESS
- else
- printf '%s\n' "$err_msg"
- return $RET_FILE_ERR
- fi
- else
- printf '%s\n' "$err_msg"
- return $RET_FAILURE
- fi
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Ask a yes/no question.
-# Note, that ^D is considered as a default answer.
-#
-# Not used.
-#
-# $1 default answer: (y|n|x)
-# $2 question: string
-#
-# returns yes / no: (0|1)
-#
-# interactive.
-#
-function ask
-{
- # -l auto-converts everything to lower case.
- local -l default="$1"
- local question="$2"
-
- while true
- do
- if [[ "$default" == "y" ]]
- then
- printf "%s [Y/n]? " "$question"
- elif [[ "$default" == "n" ]]
- then
- printf "%s [y/N]? " "$question"
- else
- printf "%s [y/n]? " "$question"
- default="x"
- fi
-
- local -l reply # -l converts to lowercase
- read reply
-
- if [[ "$reply" =~ ^y(es)?$ ]]
- then
- return 0
- elif [[ "$reply" =~ ^no?$ ]]
- then
- return 1
- elif [[ "$default" == "x" || "$reply" != "" ]]
- then
- continue
- elif [[ "$default" == "y" ]]
- then
- return 0
- else # "$default" == "n"
- return 1
- fi
- done
-}
+# Include the other files.
+. ./variables
+. ./files
+. ./write
+. ./remote
+. ./local
+. ./source
+. ./destination
main "$@"
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/source b/source
new file mode 100644
index 0000000..5bbdac6
--- /dev/null
+++ b/source
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+#
+# source - functions acting on the source host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Set up a new local mirror file tree on the source.
+#
+# requires: assert_is_dir, create_dir, replace_symlink,
+# $src_path, $src_mirror_reldir,
+# $RSYNC, ${rsync_args[@]}, ${filter_args[@]},
+# $DATEFMT,
+#
+function src_prepare
+{
+ local mirrors_dir="${src_path}/${src_mirror_reldir}"
+ local old_mirror="${mirrors_dir}/old"
+ local new_mirror="${mirrors_dir}/new"
+ local tmp_mirror="${mirrors_dir}/tmp"
+ local bakdate
+
+ assert_is_dir "$src_path" || return $?
+
+ message 'Create mirror of the source tree...'
+
+ if [[ ! -d "$mirrors_dir" ]]
+ then
+ create_dir "$mirrors_dir" || return $?
+ create_dir "$tmp_mirror" || return $?
+ ln -s 'none' "$old_mirror" || return $RET_ERROR
+ else
+ replace_symlink "$new_mirror" "$old_mirror" || return $?
+ create_dir "$tmp_mirror" || return $?
+ fi
+
+ "$RSYNC" \
+ --exclude="/${src_mirror_reldir}/***" \
+ "${rsync_args[@]}" \
+ "${filter_args[@]}" \
+ --link-dest="$src_path" \
+ "${src_path}/" "$tmp_mirror" || return $RET_ERROR
+
+ bakdate=`date +"$DATEFMT"`
+ mv "$tmp_mirror" "${mirrors_dir}/${bakdate}" || return $RET_ERROR
+ ln -s "$bakdate" "$new_mirror" || return $RET_ERROR
+
+ message 'Finished creating mirror of the source tree.'
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/variables b/variables
new file mode 100644
index 0000000..e3eb451
--- /dev/null
+++ b/variables
@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+#
+# variables.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+## CONSTANTS
+typeset -r CONFIG=/etc/rsync-backup/config
+
+typeset -r RET_TYPES='
+ RET_SUCCESS RET_FAILURE RET_BADSYNTAX RET_FILE_ERR RET_ERROR'
+typeset -r RET_SUCCESS=0
+typeset -r RET_FAILURE=1
+typeset -r RET_BADSYNTAX=2
+typeset -r RET_FILE_ERR=4
+typeset -r RET_ERROR=8
+
+typeset -r DATEFMT='%Y-%m-%d_%H%M%S'
+
+# Shell options:
+# nullglob: In case of a non-matching glob, return nothing.
+# dotglob: Include dotfiles (except . and ..) in glob matching.
+typeset -r SHOPTS='dotglob'
+typeset -r SHOPTS_AFTER_GET_ARGS='nullglob'
+
+# Always give these variables and functions to remotely executed commands,
+# as they are very commonly used.
+# These will not be listed as required.
+# Note that $host is given as argument to run_function.
+typeset -r REMOTE_DEFAULT_VARS="$RET_TYPES verbose debug host"
+typeset -r REMOTE_DEFAULT_FUNS='prefixed_write message debug error
+ debug_error'
+
+
+## Global program variables.
+typeset src src_path src_host
+typeset dest dest_path dest_host
+typeset -a filter_args
+typeset -i n
+
+
+## Variables that may be configured.
+typeset RSYNC=rsync
+
+typeset verbose=true
+typeset debug=true
+
+# Location to put mirrored trees on the source.
+typeset src_mirror_reldir='.backup'
+
+typeset -a rsync_base_args
+typeset -a rsync_mirror_args
+typeset -a rsync_transfer_args
+typeset -A bakpath
+typeset -A filter_file
+typeset filter_file_all # Applied after specific filter.
+
+# Base arguments for rsync (-rptogAXlHS, --timeout=, --numeric-ids, --info=).
+# These are used both for creation of the source mirror and the transfer of
+# that mirror to the destination. In both cases, attributes may be added.
+rsync_args=(
+ --recursive
+ --perms --times --owner --group
+ --acls --xattrs
+ --links
+ --hard-links
+ --sparse
+ --numeric-ids
+ --timeout=60
+ #-vv
+ --info=progress2
+ )
+
+source "$CONFIG"
+
+
+# Set the globabl variables $src, $dest_path and $filter_args according to
+# command line parameters and the configuration.
+#
+# $@: command line parameters
+#
+# requires the shell option nullglob to be _unset_. (for test -v to work)
+#
+function get_args
+{
+ if [ $# -eq 0 ]
+ then
+ message "Usage: $exec_name <src> [<dest>]" 2>&1
+ return $RET_BADSYNTAX
+ fi
+
+ local l_src="$1"
+ local l_dest
+
+ # Set $src_path, $src_host and $src.
+ resolve_host_path src "$l_src" || return $?
+
+ if [ $# -gt 1 ]
+ then
+ l_dest="$2"
+ elif [[ -v bakpath[$src] ]] # $# -eq 1
+ then
+ l_dest="${bakpath[$src]}"
+ else
+ error "Backup path for \"$src\" neither configured" \
+ "nor specified on command line."
+
+ return $RET_BADSYNTAX
+ fi
+
+ # Set $dest_path, $dest_host and $dest.
+ resolve_host_path dest "$l_dest" || return $?
+
+ filter_args=()
+ if [[ -v filter_file[$src] && -f "${filter_file[$src]}" ]]
+ then
+ filter_args+=("--filter=merge ${filter_file[$src]}")
+ fi
+ if [[ -v filter_file_all && -f "$filter_file_all" ]]
+ then
+ filter_args+=("--filter=merge ${filter_file_all}")
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Split host:target like rsync and convert local (!) relative paths to
+# absolute ones. This is needed at least for $src_path which is used as
+# key for associative arrays.
+#
+# The results are written to the global variables $<vtype>, $<vtype>_host
+# and $<vtype>_path, where <vtype> is either src or dest. $<vtype> gets
+# set to the original to be resolved string, unless we are local, i.e.
+# there is no host.
+#
+# Note: This is not a place for bash to shine. Perl for instance would do
+# this much more easily.
+#
+# Examples:
+# - a:b -> (a, b)
+# - a/b:c -> ('', a/b:c)
+#
+# $1: which global variables to set: (src|dest)
+# $2: path string potentially containing host prefix
+#
+# sets: ${$1}_host: host name (may be empty)
+# ${$1}_path: path (absolute, if local, i.e. ${$1}_host='')
+# ${$1}: ${$1}_path, if ${$1}_host='',
+# ${$1}_host:${$1}_path (= $2), elsewise.
+#
+function resolve_host_path
+{
+ local vtype="$1"
+ local combined="$2"
+ local host path
+
+ # Split host and path.
+ host=${combined%%:*}
+ if [[ "$host" == "$combined" || "$host" =~ '/' ]]
+ then
+ host=''
+ path="$combined"
+
+ # Resolve to absolute directory.
+ # pwd -P avoids symlinks.
+ # Do not resolve symlimks in ${path} or test for its existance.
+ # `- realpath could do that.
+ if ! [[ "$path" =~ / ]]
+ then
+ path="$(pwd -P)/${path}"
+ fi
+
+ combined="$path"
+ else
+ path="${combined#*:}"
+ fi
+
+ # Set global variables.
+ if [[ "$vtype" == 'src' ]]
+ then
+ src_host="$host"
+ src_path="$path"
+ src="$combined"
+ else # "$vtype" == 'dest'
+ dest_host="$host"
+ dest_path="$path"
+ dest="$combined"
+ fi
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/write b/write
new file mode 100644
index 0000000..081bbd2
--- /dev/null
+++ b/write
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+#
+# write - output functions.
+#
+# Copyright 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Write message, prefixed by either 'local' or the respective current host.
+# Each argument is printed on its own line.
+#
+# requires: $host (unset or empty if running on localhost)
+#
+# $1: secondary prefix
+# ${@:2}: messages: string+
+#
+function prefixed_write
+{
+ local prefix
+ local msg
+
+ test -n "$host" && prefix="${host}: "
+ test -n "$1" && prefix+="${1}: "
+
+ printf '%s%s\n' "$prefix" "$2"
+
+ for msg in "${@:3}"
+ do
+ printf '%s%s\n' "${prefix//?/ }" "$msg"
+ done
+}
+
+function message
+{
+ $verbose && prefixed_write '' "$@"
+}
+
+function debug
+{
+ $debug && prefixed_write "Debug" "$@"
+}
+
+function error
+{
+ prefixed_write "Error" "$@" 2>&1
+}
+
+function debug_error
+{
+ $debug && prefixed_write "Debug" "$@" 2>&1
+}
+
+# vi: ft=bash ts=2 sw=2 noet