aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorEinhard Leichtfuß <alguien@respiranto.de>2019-10-18 00:28:00 +0200
committerEinhard Leichtfuß <alguien@respiranto.de>2019-10-18 01:17:39 +0200
commit4e2ceb9875c81c33b09f6eaf0858b7f9591b22cc (patch)
treebe3860dc8c26b3c18a30c2a81b58e00025b98069
parent51f58d7c4c984ec8ccc5ab0a3d31d04cd39d499c (diff)
downloadrsync-backup-devel.tar.gz
rsync-backup-devel.tar.xz
Restructure, and rewrite substantial partsHEADmasterdevel
A notable problem (to me) with the former version was, that upon simple movements in the source file tree, these would in general cause duplicated data in the backup forest. This problem has been resolved by essentially tracking file movements, using hard links. Also, the code has been divided into several different files, extranous code removed, the organization of remote code execution simplified. In the process of simplification, all parts requiring direct interaction of the user with the program have been replaced. In most cases, this means that the program now just terminates with an error instead of allowing the user to confirm deletion of an unexpected (remnant) file. This may be considered a drawback, but actually these interactive options were anyways suboptimal solutions in most cases - where they occured, which was due to remnant files of aborted or failed former backups. In general, there have been a lot of changes. Which are not thoroughly documented here. Since there is often no strong relation to the old code, this is not deemed necessary, as the in-code documentation is expected to be of sufficient help. Also, there has been added a little general documentation in the README and particularly the NOTES file. It is to be noted that this new version is far from sophisticated. It has been tested with and is in use for real data, however lacks a lot of convenience. In particular, if a backup fails unexpectedly, the next backup will in the very most cases loudly fail without manual intervention. Also, the program is not able to continue the growth of a backup tree built with former versions of this program, by itself. This can however be arranged by hand.
-rw-r--r--NOTES140
-rw-r--r--README66
-rw-r--r--TODO29
-rw-r--r--destination169
-rw-r--r--files153
-rw-r--r--local123
-rw-r--r--remote92
-rwxr-xr-xrsync-backup648
-rw-r--r--source69
-rw-r--r--variables211
-rw-r--r--write68
11 files changed, 1125 insertions, 643 deletions
diff --git a/NOTES b/NOTES
new file mode 100644
index 0000000..cc84920
--- /dev/null
+++ b/NOTES
@@ -0,0 +1,140 @@
+This is intended as a short, rather high-level, documentation.
+
+
+# -- NOTATIONS -- #
+
+ Define a few notations, as used in the following.
+
+- file tree: A file tree with possible excluded subtrees residing on exactly
+ one file system. The file tree does not need to share the root
+ of that file system.
+
+
+
+# -- GENERAL -- #
+
+ This backup program makes incremental backups of a file tree, using hard
+links between differently dated backups of that file tree to reduce space
+requirements while keeping the data in a simple form.
+
+ The destination file system is usually different to the on the source file
+tree resides on, in particular, it may be on a different host. The rsync(1)
+command line program serves as backend. Therefore, at least one of the
+source and destination need to be local.
+
+ The backup process is organized in a way such that any pure file renames
+between the old and new backup directory do not cause a new file to be
+created. I.e., the two files are hard linked. This is contrary to the
+simpler method of simply exploiting rsync's '--link-dest' functionality, and
+thereby to the early versions of this program.
+
+ This is achieved by
+
+ 1. Creating a local mirror of the source tree inside itself
+ (non-recursively, of course), using hard links. In common backup
+ operation, there are always at least two such mirrors: an obsolete one,
+ a remnant of the last backup's creation, featuring the same structure
+ as the last backup as stored in the destination file tree, and the newly
+ created mirror. By transitivity, the two mirrors share hard links. See
+ also APPENDIX: NOTE-1.
+
+ 2. Copying the pair of these two mirrors to the destination tree, thereby
+ referencing the old mirror to the (identical!) old mirror in the
+ destination tree. Since the two mirrors are inter-linked, and the
+ referencing allows for linking the old mirror to the (old) latest backup,
+ the new mirror (and very latest backup) is effectively linked to the
+ old latest backup. See also APPENDIX: NOTE-2.
+
+
+ The chosen procedure has some side effects.
+
+ * Backups are available on the source host (however they likely diverge from
+ the actual backups, as the former are linked to the live file tree and
+ hence change upon in-place modification). See also APPENDIX: NOTE-1.
+ `- Note that, currently, neither in the source nor in the destination tree,
+ backups (resp. mirrors) are deleted. In the source tree this might be
+ more desired than in the destination tree.
+
+ * The local mirror creation is much faster than the transfer to the backup
+ medium, in particular, if it is remote. Therefore, the state, excluding
+ the aforementioned in-place modifications, is closer to consistency with a
+ single point in time. One might argue though, that precisely these
+ in-place modifications cause the window of modifications to be in fact
+ widened to the adjunction of the mirroring and the transfer.
+
+
+ There are also some clear (but probably not grave) drawbacks to the simpler
+method.
+
+ * There is one more step involved, thereby adding complexity.
+
+ * Both the source and destination file tree need to reside on respectively
+ one single file system. As a consequence, this program needs to be
+ configured for each file system containing data to be backed up. This
+ could in theory be done in the program itself, but seems not worth the
+ effort.
+
+ * A little additional space is needed on the source file system, however
+ only for new file links and directories, not for new actual regular files.
+ `- In particular though, the source file system needs to be writable. This
+ problem may possibly be circumvented using bind mounts.
+
+
+
+# -- REMOTE METHOD CALLS -- #
+
+ This program essentially wraps rsync, however performs a few additional,
+useful functions, some of which need to be executed on the source or
+destination host, irrespective of whether that is local or not. In order to
+allow for this, there exist wrapper methods, namely 'run_function' and
+'run_command', allowing to execute a local bash function on a remote host.
+Methods to be executed either on the source or the destination are located in
+the source files 'source' and 'destination' and annotated with environment
+dependencies ('requires'). See also the 'remote' source file.
+
+
+
+# -- APPENDIX -- #
+
+# NOTE-1: The old mirror.
+ As the old mirror is a remnant of the last backup it features exactly the
+ same file tree as that last backup - by labels, not by content. This is
+ due to this old mirror being (purposefully) linked to the live file tree,
+ in which files may be modified in place, which due to the hard links also
+ appear in the old mirror.
+
+
+# NOTE-2: Avoid propagation of errors on the old source mirror.
+ The simple way to do the transfer from the source to the destination tree
+ may seem to use the old latest backup on the destination as part of the
+ target, when referencing the old mirror to it.
+
+ I.e., sync (old_mirror, new_mirror) -> (old_latest_backup, new_backup).
+ This is however, not a good idea, since as explicated in NOTE-1, old_mirror
+ may change and hence the above operation would change old_latest_backup.
+ Since this is not desired, old_latest_backup is instead only used as
+ link-dest for rsync, which unfortunately causes a superfluous instance of
+ old_latest_backup to be created in the destination tree. This is simply
+ deleted in the end.
+
+
+# NOTE-3: Imperfection.
+ The chosen solution is far from perfect. In particular, the backups are
+ far from being atomar. Apparently, the Btrfs makes this easier [0].
+ Another alternative might be to use an overlayFS to record new writes.
+
+ Further, in order to at least get closer to atomicity, a full copy, made
+ on and of the source tree, could be done before actual transferral. This
+ would however require a lot of space and also only likely be beneficial, if
+ the transfer is non-locally.
+
+ Also, the extranously created copy of the source's old mirror on the
+ destination is disturbing. I do however not know of way to avoid this,
+ unless not relying on the rsync(1) command line program as backend.
+
+
+# Links
+[0] https://btrfs.wiki.kernel.org/index.php/Incremental_Backup
+
+
+vi: ai et tw=77 fo+=t
diff --git a/README b/README
index 1287b72..66ba148 100644
--- a/README
+++ b/README
@@ -1,23 +1,77 @@
-- in work -
+Introduction
+------------
+
+ rsync-backup is a frontend to rsync (with a few extras) to allow for the
+creation of incremental backups, using hard links. It notably is tracking
+movements, that is a file moved (but unchanged) in the source file tree will
+not be stored twice for two corresponding backups - before and after the
+movement.
Dependencies
------------
-Most dependencies are required locally and remotely, as a huge part of the
+ Most dependencies are required locally and remotely, as a huge part of the
script may be executed remotely.
- bash>=4.2 (test -v)
- rsync
- ssh
-- coreutils (realpath, date)
+- coreutils
- find
+Installation
+------------
+
+ Put all the source files in one directory. It may be desirable to set up
+a symbolic link in e.g. /usr/local/bin, or a shell alias.
+
+
+Usage
+-----
+
+ Execute 'rsync-backup' either with one argument, specifying the source
+directory, requiring that to be configured (see Configuration) or two
+arguments, where the second one specifies the destination.
+
+ It is highly recommended to configure the respective directories, not least
+to configure to be excluded parts of the file tree (e.g., /dev, /mnt).
+
+ It is possible to use a remote location for one of the source and
+destination.
+
+ Examples:
+$ rsync-backup /home # requires prior configuration
+$ rsync-backup / user@server:/backup
+
+
+Configuration
+-------------
+
+ Configuration may be done in the file '/etc/rsync-backup/config'. See the
+source file 'variables' for information on which variables can be set.
+
+
+Documentation
+-------------
+
+As of now, documentation on the usage is found only here.
+
+The file NOTES provides an overview of the ideas and the (inner) working of
+this program.
+
+
Notes
-----
-Symbolic links are simply copied, thus linking to the backed-up filetree, not
-the backup-filetree itself.
+ IMPORTANT: Both the source and destination file tree should reside on a
+single file system. It is highly recommended to exclude (config:
+filter_file) other mounted file systems. Otherwise, assuming the backup
+process does not fail, each new backup is expected to take the full space for
+such mounted subdirectories.
+
+ Symbolic links are simply copied, thus linking to the backed-up filetree,
+not the backup-filetree itself.
-Dates are always calculated on the local host.
+ Dates are always calculated on the source host.
diff --git a/TODO b/TODO
index caa934b..142a7bb 100644
--- a/TODO
+++ b/TODO
@@ -1,4 +1,8 @@
# -- BUGS -- #
+- /path/ is not recognized as configured, when only /path explicitly is
+ configured.
+- After a failure, the .backup directory is in a bad state (contains backup
+ data labeled as old, however not related to the destination).
# -- TODO -- #
- For ssh, set up one single connection (maybe configurable?).
@@ -6,13 +10,11 @@
-oControlMaster=yes \
-oControlPersist $remote /bin/true
`- ssh -oControlPath=/tmp/root-readable/something.sock
-- Replace echo by printf(-functions).
- Set up default configuration file.
- [consider] When installing, copy the config file to both /etc and
/usr/share, such that sourcing from the latter location removes the need to
specify default options in the script itself.
- Test for read/write access at some point.
-- Better error reporting.
- Use autotools.
- Per dirpath rsync_args.
- Allow for resuming of backups.
@@ -21,7 +23,7 @@
`- Not a good idea if a notable amount of time passed.
`- [question] What happens in case of partially transferred files?
`- --delete-during, --delete-excluded
-- Shell completion of configured backup paths.
+- [consider] Shell completion of configured backup paths.
- Better error handling on failure (of rsync).
- Use ask().
`- Allow for configuration of default answer.
@@ -29,6 +31,12 @@
- [consider] Get date on source host.
- Named backups to allow for different destinations per source (and shorter
names).
+- Test suite.
+- Check for several file systems in a tree (and error upon locating such).
+- [consider] Add fallback if no local mirror of the source file tree is
+ available.
+- Only store the last mirror locally, but date it.
+- [consider] --one-file-system as default option.
# -- PROBLEMS -- #
- rsync bug: https://bugzilla.samba.org/show_bug.cgi?id=13445
@@ -56,3 +64,18 @@
`- For now, -v or -vv in rsync_args should work fine.
- non-intercative option.
- rsync_network_options (when remote src or dest, e.g. for --compress).
+- Write metadata.
+- Use wrappers for the functions possibly non locally executed.
+- Compare src:work/$old_date and dest:$old_date for consistency.
+- Run script instead of function remotely
+ `- ssh host 'bash -s' -- < script_file args
+ `- In case of dependencies, the otherwise sourced files could just be
+ included in the stdin of ssh.
+ `- cat script_file dep_file | ssh host 'bash -s' -- args
+- Use stdout of functions to return a string value and stderr for everything
+ else.
+ `- Optional: Prefixes determining whether normal or error.
+ `- Convert at the end (remove prefix and pipe to stderr / stdout).
+- Only keep the respectively latest backup on the source.
+ `- Or the last two, which would allow to reuse the second-last as new
+ mirror (for a probably negligible gain, if any).
diff --git a/destination b/destination
new file mode 100644
index 0000000..4829838
--- /dev/null
+++ b/destination
@@ -0,0 +1,169 @@
+#!/usr/bin/env bash
+#
+# destination - functions acting on the destination host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Setup working directory on the destination files system.
+#
+# requires: create_new_dir
+#
+function dest_prepare_initial
+{
+ create_new_dir "$dest_path" || return $?
+ create_new_dir "${dest_path}/work" || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Setup working directory on the destination file system, check for too new
+# (future) backups and refresh the 'latest' symlink.
+#
+# requires: assert_is_dir, dest_list_backup_dirs, create_new_dir
+# $dest_path
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function dest_prepare_incremental
+{
+ local work_dir="${dest_path}/work"
+ local old_date="$1"
+ local new_date="$2"
+ local latest
+
+ assert_is_dir "$dest_path" || return $?
+
+ latest="$(dest_list_backup_dirs | tail -1)"
+
+ if [[ "$old_date" != "$latest" ]]
+ then
+ error "Dates of last backup do not match on source and destination."
+ debug_error "source: ${old_date} | dest: ${latest}"
+ return $RET_FILE_ERR
+ fi
+
+ if ! [[ "$latest" < "$new_date" ]]
+ then
+ error "Latest backup on destination not older than source."
+ return $RET_FILE_ERR
+ fi
+
+ # Refresh 'latest' symlink in case it was obsoleted or corrupted.
+ ln -sfT "$latest" "${dest_path}/latest" || return $RET_ERROR
+
+ create_new_dir "$work_dir" || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Finalize the backup creation on the destination host.
+#
+# requires: dest_refresh_symlinks
+# `-> assert_is_dir, dest_list_backup_dirs,
+# replace_symlink_target, create_dir
+# $dest_path (self, <- dest_refresh_symlinks),
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function dest_finalize
+{
+ local old_date="$1"
+ local new_date="$2"
+
+ message "Finish installment of new backup..."
+ mv -T "${dest_path}/work/${new_date}" "${dest_path}/${new_date}" \
+ || return $RET_ERROR
+
+ message "Remove auxiliary directory tree..."
+ rm -rf "${dest_path}/work/${old_date}" || return $RET_ERROR
+ rmdir "${dest_path}/work" || return $RET_ERROR
+
+ dest_refresh_symlinks || return $?
+
+ message 'Successfully finished installment of new backup.'
+}
+
+
+# Print existing backup directories.
+# Must be run on destination host.
+#
+# requires: $dest_path
+#
+function dest_list_backup_dirs
+{
+ find "$dest_path" -mindepth 1 -maxdepth 1 -type d \
+ -regextype posix-extended -regex \
+ '.*/[0-9]{4}-[0-1][0-9]-[0-3][0-9]_[0-2][0-9]([0-5][0-9]){2}' \
+ -printf '%f\n' \
+ | LC_ALL=C sort
+
+ if [[ ${PIPESTATUS[0]} -eq 0 && ${PIPESTATUS[1]} -eq 0 ]]
+ then
+ return $RET_SUCCESS
+ else
+ return $RET_ERROR
+ fi
+}
+
+
+# Create symbolic links to backup directories.
+#
+# requires: assert_is_dir, dest_list_backup_dirs, replace_symlink_target,
+# create_dir
+# $dest_path
+#
+function dest_refresh_symlinks
+{
+ local -i i
+ local -i n
+ local -a backup_dirs
+ local link
+ local ret
+
+ assert_is_dir "$dest_path" || return $?
+
+ backup_dirs=( $(dest_list_backup_dirs || return $?) )
+ n=${#backup_dirs[@]}
+
+ replace_symlink_target "${dest_path}/latest" "${backup_dirs[n-1]}" \
+ || return $?
+
+ ## by_number
+ numdir="${dest_path}/by_number"
+ create_dir "$numdir" || return $?
+
+ # Remove old symlinks.
+ find "$numdir" -mindepth 1 -maxdepth 1 -type l -regextype posix-extended \
+ -regex '.*/[0-9]*' -delete || return $RET_ERROR
+
+ for (( i = 0; i < n; i++ ))
+ do
+ ln -s "../${backup_dirs[i]}" "$(printf "%s/%.${#n}u" "$numdir" $i)" \
+ || return $RET_FILE_ERR
+ done
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/files b/files
new file mode 100644
index 0000000..85640e7
--- /dev/null
+++ b/files
@@ -0,0 +1,153 @@
+#!/usr/bin/env bash
+#
+# files - functions acting on files.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Create directory, unless it exists already. Returns non-zero, if the
+# named file exists however is not a directory.
+#
+# $1: directory: path
+#
+function create_dir
+{
+ local dir="$1"
+
+ if [[ ! -e "$dir" && ! -L "$dir" ]]
+ then
+ mkdir "$dir" || return $RET_ERROR
+ elif [[ ! -d "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" is not a directory."
+ return $RET_FILE_ERR
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Create a new directory, assuring it did not exist before.
+#
+# $1: directory: path
+#
+function create_new_dir
+{
+ local dir="$1"
+
+ if [[ -e "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" exists already."
+ return $RET_FILE_ERR
+ fi
+
+ mkdir "$dir" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Verify the specified file is a directory.
+#
+# $1: directory: path
+#
+function assert_is_dir
+{
+ local dir="$1"
+
+ if [[ ! -d "$dir" || -L "$dir" ]]
+ then
+ error "\"$dir\" is not a directory."
+ return $RET_FILE_ERR
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Replace symbolic link $2 by symbolic link $1. $2 may not exist.
+#
+# $1: source file name: path
+# $2: target file name: path
+#
+function replace_symlink
+{
+ local src="$1"
+ local tgt="$2"
+
+ if [[ ! -L "$src" ]]
+ then
+ error "\"$src\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ if [[ ! -L "$tgt" && -e "$tgt" ]]
+ then
+ error "\"$tgt\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ mv -T "$src" "$tgt" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Replace symbolic link $1's target by $2. $1 may not exist.
+#
+# $1: link file: path
+# $2: link target: path
+#
+function replace_symlink_target
+{
+ local link="$1"
+ local tgt="$2"
+
+ if [[ ! -L "$link" && -e "$link" ]]
+ then
+ error "\"$link\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ ln -sfT "$tgt" "$link" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+
+# Remove a symbolic link.
+#
+# $1: link: path
+#
+function remove_symlink
+{
+ local file="$1"
+
+ if [[ ! -L "$file" ]]
+ then
+ error "\"$file\" is not a symbolic link."
+ return $RET_FILE_ERR
+ fi
+
+ rm "$file" || return $RET_ERROR
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/local b/local
new file mode 100644
index 0000000..37b2574
--- /dev/null
+++ b/local
@@ -0,0 +1,123 @@
+#!/usr/bin/env bash
+#
+# local - functions acting on the local host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+function main
+{
+ local src_old_date
+ local src_new_date
+ local src_mirrors_path
+
+ shopt -s $SHOPTS
+
+ # Set $src, $dest_path and $filter_args.
+ get_args "$@" || return $?
+
+ shopt -s $SHOPTS_AFTER_GET_ARGS
+
+ run_function "$src_host" src_prepare \
+ 'assert_is_dir create_dir replace_symlink' \
+ 'src_path src_mirror_reldir RSYNC rsync_args[@] filter_args[@] DATEFMT' \
+ || return $?
+
+ src_mirrors_path="${src_path}/${src_mirror_reldir}"
+ src_old_date="$(
+ run_command "$src_host" readlink "${src_mirrors_path}/old")"
+ src_new_date="$(
+ run_command "$src_host" readlink "${src_mirrors_path}/new")"
+
+ if [[ "$src_old_date" == 'none' ]]
+ then
+ run_function "$dest_host" dest_prepare_initial 'create_new_dir' '' \
+ || return $?
+
+ transfer_initial_backup "$src_new_date" || return $?
+ else
+ run_function "$dest_host" dest_prepare_incremental \
+ 'assert_is_dir dest_list_backup_dirs create_new_dir' 'dest_path' \
+ "$src_old_date" "$src_new_date" \
+ || return $?
+
+ transfer_incremental_backup "$src_old_date" "$src_new_date" || return $?
+ fi
+
+ run_function "$dest_host" dest_finalize \
+ 'dest_refresh_symlinks
+ assert_is_dir dest_list_backup_dirs replace_symlink_target
+ create_dir' \
+ 'dest_path' \
+ "$src_old_date" "$src_new_date" \
+ || return $?
+
+ return $RET_SUCCESS
+}
+
+
+# Transfer the very first backup from source to destination. To be run after
+# src_prepare and dest_prepare.
+#
+# $1: date of the backup: date in $DATEFMT
+#
+function transfer_initial_backup
+{
+ local date="$1"
+
+ message "Transfer new (initial) backup..."
+
+ "$RSYNC" "${rsync_args[@]}" \
+ "${src}/${src_mirror_reldir}/${date}" \
+ "${dest}/work" \
+ || return $RET_ERROR
+
+ message "Finished transfer."
+
+ return $RET_SUCCESS
+}
+
+
+# Transfer new backup from source to destination. To be run after
+# src_prepare and dest_prepare.
+#
+# $1: old date: date in $DATEFMT
+# $2: new date: date in $DATEFMT
+#
+function transfer_incremental_backup
+{
+ local old_date="$1"
+ local new_date="$2"
+
+ message "Transfer new (incremental) backup..."
+
+ # Note: Use --link-dest instead of making the old backup part of the target
+ # directory, since src/old_date may have changed due to in-place
+ # modifications.
+ "$RSYNC" "${rsync_args[@]}" --link-dest='..' \
+ "${src}/${src_mirror_reldir}"/{"$old_date","$new_date"} \
+ "${dest}/work" \
+ || return $RET_ERROR
+
+ message "Finished transfer."
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/remote b/remote
new file mode 100644
index 0000000..1b04197
--- /dev/null
+++ b/remote
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+#
+# remote - execute functions remotely.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Run command remotely.
+#
+# $1: host: name
+# ${@:2}: command with arguments: string+
+#
+function run_command
+{
+ local host="$1"
+ shift
+
+ if [[ -z "$host" ]]
+ then
+ "$@"
+ else
+ # Use ssh to run command remotely.
+ # Make sure to properly escape the strings.
+ ssh "$host" "shopt -s ${SHOPTS}; $(printf "'%s' " "$@")"
+ fi
+
+ return $?
+}
+
+
+# Run a function on a remote host.
+# If the host parameter is empty, run locally.
+#
+# $1: host: name
+# $2: function to run: name
+# $3: other functions to export: [name (. ' ' . name)*]
+# $4: variables to export: [name (. '[@]')? (. ' ' . name (. '[@]')?)*]
+# ${@:5}: function arguments: string+
+#
+function run_function
+{
+ local host="$1"
+ local fun=$2
+ local funs="$fun $3 $REMOTE_DEFAULT_FUNS"
+ local vars="$4 $REMOTE_DEFAULT_VARS"
+ local vars_set
+
+ shift 4
+
+ if [[ -z "$host" ]]
+ then
+ $fun "$@"
+ else
+ # Create string to export variables.
+ vars_set=
+
+ # Avoid surprises of Bash's globbing (due to brackets in, e.g., a[@]).
+ set -o noglob
+
+ for var in $vars
+ do
+ vars_set+="$(eval echo "\${${var}@A}"); "
+ done
+
+ set +o noglob
+
+ # Use ssh to run command remotely.
+ # Make sure to properly escape the strings.
+ ssh "$host" "shopt -s ${SHOPTS}; ${vars_set} $(typeset -f $funs);" \
+ "$fun $(printf "'%s' " "$@")"
+ fi
+
+ return $?
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/rsync-backup b/rsync-backup
index 0a7df33..86bff28 100755
--- a/rsync-backup
+++ b/rsync-backup
@@ -21,642 +21,22 @@
#
-typeset -r exec_name="${0##*/}"
+typeset -r full_exec_name="$(realpath "$0")"
+typeset -r exec_name="${full_exec_name##*/}"
+typeset -r source_code_dir="${full_exec_name%/*}"
+cd "$source_code_dir"
-# CONSTANTS
-typeset -r CONFIG=/etc/rsync-backup/config
-typeset -r RET_TYPES='
- RET_SUCCESS RET_FAILURE RET_BADSYNTAX RET_FILE_ERR RET_ERROR'
-typeset -r RET_SUCCESS=0
-typeset -r RET_FAILURE=1
-typeset -r RET_BADSYNTAX=2
-typeset -r RET_FILE_ERR=4
-typeset -r RET_ERROR=8
-typeset -r DATEFMT='%Y-%m-%d_%H%M%S'
-
-# Shell options:
-# nullglob: In case of a non-matching glob, return nothing.
-# dotglob: Include dotfiles (except . and ..) in glob matching.
-typeset -r SHOPTS='dotglob'
-typeset -r SHOPTS_AFTER_GET_ARGS='nullglob'
-
-
-# Global program variables.
-typeset src src_path src_host
-typeset dest dest_path dest_host
-typeset -a filter_args
-typeset -i n
-typeset bakdate
-
-
-# Variables that may be configured.
-typeset RSYNC=rsync
-typeset -a rsync_args
-typeset -a rsync_inc_args
-typeset -A bakpath
-typeset filter_file_all # Applied after specific filter.
-typeset -A filter_file
-
-# Default arguments for rsync (-rptogAXlHS, --timeout=, --info=).
-rsync_args=(
- --recursive
- --perms --times --owner --group
- --acls --xattrs
- --links
- --hard-links
- --sparse
- --timeout=60
- #-vv
- --info=progress2
- )
-
-# Supplemental arguments for rsync; only used when an actual incremental
-# update is performed, i.e. when --link-dest is used.
-rsync_inc_args=(
- --fuzzy --fuzzy
- )
-
-filter_file_all=/etc/rsync-backup/filter
-
-source "$CONFIG"
-
-
-# RSYNC args.
-# basic args: --recursive --perms --times --owner --group --links
-# extra args: --sparse --acls --xattrs --hard-links --timeout=60
-# --fuzzy --fuzzy
-# --filter="merge <file>"
-# likely: --compress
-# --partial{,dir=DIR}
-# --progress
-# possibly: --devices --specials
-# --max-size=SIZE-OF-FILE
-# --one-file-system
-# --log-file=FILE --log-file-format=FORMAT
-# --human-readable
-# unlikely: --omit-{dir,link}-times --update (!--inplace) --delete
-# --exclude
-# great: --link-dest=DIR (timestamps?)
-# testing: --verbose --dry-run (-vn)
-# interesting: --sockopts --itemize-changes --out-format=FORMAT
-# --stats
-
-function main
-{
- shopt -s $SHOPTS
-
- # Set $src, $dest_path and $filter_args.
- get_args "$@" || return $?
-
- shopt -s $SHOPTS_AFTER_GET_ARGS
-
- # Verify $src_path exists.
- run_function "$src_host" src_check_path '' 'src_path src' || return $?
-
- # Make sure $dest_path exists.
- run_function "$dest_host" dest_check_path '' 'dest_path dest' || return $?
-
- # Populate the global $dirs array.
- dirs=( $(
- run_function "$dest_host" dest_find_existing_backup_dirs '' 'dest_path' \
- || return $?
- ) )
-
- n=${#dirs[@]}
-
- run_function "$dest_host" dest_prepare 'dest_clear_file' \
- 'dest dirs[@] n dest_host dest_path' "$(date +$DATEFMT)" \
- || return $?
-
- perform_backup || return $?
-
- run_function "$dest_host" dest_finalize \
- 'dest_refresh_symlinks dest_clear_file' \
- 'dest_host dest_path bakdate n' \
- || return $?
-
- dest_refresh_symlinks
-
- return $RET_SUCCESS
-}
-
-
-# Set the globabl variables $src, $dest_path and $filter_args according to
-# command line parameters and the configuration.
-#
-# $@: command line parameters
-#
-# requires the shell option nullglob to be _unset_. (for test -v to work)
-#
-function get_args
-{
- if [ $# -eq 0 ]
- then
- echo "Usage: $exec_name <src> [<dest>]"
- return $RET_BADSYNTAX
- fi
-
- local l_src="$1"
- local l_dest
-
- # Set $src_path, $src_host and $src.
- resolve_host_path src "$l_src" || return $?
-
- if [ $# -gt 1 ]
- then
- l_dest="$2"
- elif test -v bakpath[$src] # $# -eq 1
- then
- l_dest="${bakpath[$src]}"
- else
- echo "Error: Backup path for $src neither configured"
- echo " nor specified on command line."
-
- return $RET_BADSYNTAX
- fi
-
- # Set $dest_path, $dest_host and $dest.
- resolve_host_path dest "$l_dest" || return $?
-
- filter_args=()
- if test -v filter_file[$src] && test -f "${filter_file[$src]}"
- then
- filter_args+=("--filter=merge ${filter_file[$src]}")
- fi
- if test -v filter_file_all -a -f "$filter_file_all"
- then
- filter_args+=("--filter=merge ${filter_file_all}")
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Split host:target like rsync and convert local (!) relative paths to
-# absolute ones. This is needed at least for $src_path which is used as
-# key for associative arrays.
-#
-# The results are written to the global variables $<vtype>, $<vtype>_host
-# and $<vtype>_path, where <vtype> is either src or dest. $<vtype> gets
-# set to the original to be resolved string, unless we are local, i.e.
-# there is no host.
-#
-# Note: This is not a place for bash to shine. Perl for instance would do
-# this much more easily.
-#
-# Examples:
-# - a:b -> (a, b)
-# - a/b:c -> ('', a/b:c)
-#
-# $1: which global variables to set: (src|dest)
-# $2: path string potentially containing host prefix
-#
-# sets: ${$1}_host: host name (may be empty)
-# ${$1}_path: path (absolute, if local, i.e. ${$1}_host='')
-# ${$1}: ${$1}_path, if ${$1}_host='',
-# ${$1}_host:${$1}_path (= $2), elsewise.
-#
-function resolve_host_path
-{
- local vtype="$1"
- local combined="$2"
- local host path
-
- # Split host and path.
- host=${combined%%:*}
- if [[ "$host" == "$combined" || "$host" =~ '/' ]]
- then
- host=''
- path="$combined"
-
- # Resolve to absolute directory.
- # pwd -P avoids symlinks.
- # Do not resolve symlimks in ${path} or test for its existance.
- # `- realpath could do that.
- if ! [[ "$path" =~ / ]]
- then
- path="$(pwd -P)/${path}"
- fi
-
- combined="$path"
- else
- path="${combined#*:}"
- fi
-
- # Set global variables.
- if [[ "$vtype" == 'src' ]]
- then
- src_host="$host"
- src_path="$path"
- src="$combined"
- else # "$vtype" == 'dest'
- dest_host="$host"
- dest_path="$path"
- dest="$combined"
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Run a function on a remote host.
-# If the host parameter is empty, run locally.
-#
-# $1: host: name
-# $2: function to run: name
-# $3: other functions to export: [name (. ' ' . name)*]
-# $4: variables to export: [name (. '[@]')? (. ' ' . name (. '[@]')?)*]
-# ${@:5}: function arguments: string+
-#
-function run_function
-{
- local host="$1"
- local fun=$2
- local funs="$fun $3"
- local vars="$4"
- local vars_set
-
- shift 4
-
- if [[ -z "$host" ]]
- then
- $fun "$@"
- else
- # Create string to export variables.
- vars_set=
- for var in $RET_TYPES $vars
- do
- vars_set+="$(eval echo \${${var}@A}); "
- done
-
- # Use ssh to run command remotely.
- # Make sure to properly escape the strings.
- ssh "$host" "shopt -s ${SHOPTS}; ${vars_set} $(typeset -f $funs);" \
- "$fun $(printf "'%s' " "$@")"
- fi
-
- return $?
-}
-
-
-# Run command remotely.
-#
-# Not used.
-#
-# $1: host: name
-# ${@:2}: command with arguments: string+
-#
-function do_thing
-{
- local host="$1"
- shift
-
- if [[ -z "$host" ]]
- then
- "$@"
- else
- # Use ssh to run command remotely.
- # Make sure to properly escape the strings.
- ssh "$host" "shopt -s ${SHOPTS}; $(printf "'%s' " "$@")"
- fi
-
- return $?
-}
-
-
-# Check if $src exists.
-# Must be run on source host.
-#
-# requires: $src_path, $src
-#
-function src_check_path
-{
- if ! test -d "$src_path"
- then
- echo "Source (\"$src\") does not exist."
- return $RET_FILE_ERR
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Check if $dest_path exists and try to create it if necessary.
-# Must be run on destination host.
-#
-# requires: $dest_path, $dest
-# interactive.
-#
-function dest_check_path
-{
- local -l reply
-
- if ! test -d "$dest_path"
- then
- echo "Target (\"$dest\") does not exist."
- echo -n "Do you want do create it [y/N]? "
- read reply
- if ! [[ "$reply" =~ ^y(es)?$ ]]
- then
- return $RET_FAILURE
- fi
- echo
-
- if ! mkdir -p "$dest_path"
- then
- echo "Backup creation has failed."
- echo "\"$dest\" could not be created."
- return $RET_FILE_ERR
- fi
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Print existing backup directories.
-# Must be run on destination host.
-#
-# requires: $dest_path
-#
-function dest_find_existing_backup_dirs
-{
- find "$dest_path" -mindepth 1 -maxdepth 1 -type d \
- -regextype posix-extended -regex \
- '.*/[0-9]{4}-[0-1][0-9]-[0-3][0-9]_[0-2][0-9]([0-5][0-9]){2}.backup' \
- -printf '%f\n' \
- | LC_ALL=C sort
-
- if [[ ${PIPESTATUS[0]} -eq 0 && ${PIPESTATUS[1]} -eq 0 ]]
- then
- return $RET_SUCCESS
- else
- return $RET_ERROR
- fi
-}
-
-
-# Prepare destination for the backup process.
-# Must be run on destination host.
-#
-# requires: $dirs[@], $n
-# $dest_host (self, <- dest_clear_file)
-# $dest_path
-#
-# $1: current date on local host in format $DATEFMT
-#
-# interactive (<- dest_clear_file)
-#
-function dest_prepare
-{
- local current_date="$1"
-
- # Check for dates in the future, which might mess with the yet to be
- # determined new backup's date.
- if [[ $n -gt 0 ]]
- then
- local last_date=${dirs[n-1]%.backup}
-
- if [[ "$current_date" == "$last_date" ]]
- then
- echo "Last backup was run in the same second."
- echo "Exiting."
- return $RET_FAILURE
- elif
- [[ "$current_date" < "$last_date" ]]
- then
- echo "Last backup lies in the future. Fix your clocks."
- echo "Exiting."
- return $RET_FAILURE
- fi
- fi
-
- dest_clear_file "${dest_path}/new.backup" "Backup creation has failed." \
- || return $?
-
- return $RET_SUCCESS
-}
-
-
-# Perform the actual backup.
-# To be run locally.
-#
-# requires: $DATEFMT,
-# $dest_path, $dest, $src_path, $src,
-# $n, $dirs[@],
-# $rsync_args[@], $rsync_inc_args[@], $filter_args[@]
-#
-# sets: $bakdate
-#
-function perform_backup
-{
- bakdate=`date +"$DATEFMT"`
-
- printf 'Starting backup nr. %u.\n' "$n"
- if [ $n -eq 0 ]
- then
- # Perform first backup, not incremental.
- $RSYNC "${rsync_args[@]}" \
- "${filter_args[@]}" \
- "$src"/ "${dest}/new.backup" \
- || return $RET_ERROR
- else
- # Perform incremental backup on the basis of the last.
- $RSYNC "${rsync_args[@]}" \
- "${rsync_inc_args[@]}" --link-dest="../${dirs[n-1]}" \
- "${filter_args[@]}" \
- "$src"/ "${dest}/new.backup" \
- || return $RET_ERROR
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Finalize the backup creation on the destination host.
-#
-# requires: $dest_host (<- dest_refresh_symlinks),
-# $dest_path (self, <- dest_refresh_symlinks),
-# $bakdate (self, <- dest_refresh_symlinks),
-# $n (self, <- dest_refresh_symlinks),
-#
-# interactive (<- dest_refresh_symlinks)
-#
-function dest_finalize
-{
- mv "${dest_path}/new.backup" "${dest_path}/${bakdate}.backup" \
- || return $RET_FILE_ERR
-
- if
- dest_refresh_symlinks
- then
- printf '\nCreating of backup nr. %u successfully executed.\n' "$n"
- return $RET_SUCCESS
- else
- local ret=$?
- printf '\nCreating of backup nonetheless successfully executed.\n'
- return $ret
- fi
-}
-
-
-# Create symbolic links to backup directories.
-# Must be run on the destination host.
-#
-# requires: $dest_host, $dest_path, $bakdate, $n
-# interactive (<- dest_clear_file)
-#
-function dest_refresh_symlinks
-{
- local -i i
- local host_prefix="${dest_host}${dest_host:+:}"
- local linkdir ret
-
- linkdir="${dest_path}/latest"
- if test -L "$linkdir"
- then
- rm "$linkdir"
- ln -s "${bakdate}.backup" "$linkdir" || return $RET_ERROR
- elif
- dest_clear_file "$linkdir" "Creation of softlink \"latest\" failed." \
- || return $?
- then
- ln -s "${bakdate}.backup" "$linkdir" || return $RET_ERROR
- fi
-
- # by_number
- numdir="${dest_path}/by_number"
- if ! test -d "$numdir"
- then
- dest_clear_file "$dest_host" "$numdir" \
- "Setting up numbered softlinks failed." || return $?
-
- mkdir "$numdir" || return $RET_ERROR
- fi
-
- # Remove old symlinks.
- find "$numdir" -mindepth 1 -maxdepth 1 -type l -regextype posix-extended \
- -regex '.*/[0-9]*' -delete || return $RET_ERROR
-
- # TODO: Simplify.
- for file in "$numdir"/*
- do
- echo $file
- dest_clear_file "$file" "Setting up numbered softlinks failed." \
- || return $?
- done
-
- # Set last dir on the new one.
- dirs[$n]="${bakdate}.backup"
-
- for (( i = 0; i <= n; i++ ))
- do
- ln -s "../${dirs[i]}" "$(printf "%s/%.${#n}u" "$numdir" $i)"
- done
-
- return $RET_SUCCESS
-}
-
-
-# Make sure a file does not exist on the destination host.
-# Must be run on destination host.
-#
-# $1: file: path
-# $2: error message: string
-#
-# requires: $dest_host
-# interactive.
-#
-function dest_clear_file
-{
- local file_path="$1"
- local err_msg="$2"
- local file="${dest_host}${dest_host:+:}${file_path}"
-
- local -l reply
-
- if test -e "$file_path" -o -L "$file_path"
- then
- printf "%s %s\n %s\n%s\n" \
- "It seems like the last backup process failed or you have created" \
- "the file" "$file" "manually."
-
- echo -n "Do you want to delete it [y/N]? "
- read reply
-
- if [[ "$reply" =~ ^y(es)?$ ]]
- then
- printf "Deleting.\n"
-
- if rm -rf "$file_path"
- then
- return $RET_SUCCESS
- else
- printf '%s\n' "$err_msg"
- return $RET_FILE_ERR
- fi
- else
- printf '%s\n' "$err_msg"
- return $RET_FAILURE
- fi
- fi
-
- return $RET_SUCCESS
-}
-
-
-# Ask a yes/no question.
-# Note, that ^D is considered as a default answer.
-#
-# Not used.
-#
-# $1 default answer: (y|n|x)
-# $2 question: string
-#
-# returns yes / no: (0|1)
-#
-# interactive.
-#
-function ask
-{
- # -l auto-converts everything to lower case.
- local -l default="$1"
- local question="$2"
-
- while true
- do
- if [[ "$default" == "y" ]]
- then
- printf "%s [Y/n]? " "$question"
- elif [[ "$default" == "n" ]]
- then
- printf "%s [y/N]? " "$question"
- else
- printf "%s [y/n]? " "$question"
- default="x"
- fi
-
- local -l reply # -l converts to lowercase
- read reply
-
- if [[ "$reply" =~ ^y(es)?$ ]]
- then
- return 0
- elif [[ "$reply" =~ ^no?$ ]]
- then
- return 1
- elif [[ "$default" == "x" || "$reply" != "" ]]
- then
- continue
- elif [[ "$default" == "y" ]]
- then
- return 0
- else # "$default" == "n"
- return 1
- fi
- done
-}
+# Include the other files.
+. ./variables
+. ./files
+. ./write
+. ./remote
+. ./local
+. ./source
+. ./destination
main "$@"
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/source b/source
new file mode 100644
index 0000000..5bbdac6
--- /dev/null
+++ b/source
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+#
+# source - functions acting on the source host.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Set up a new local mirror file tree on the source.
+#
+# requires: assert_is_dir, create_dir, replace_symlink,
+# $src_path, $src_mirror_reldir,
+# $RSYNC, ${rsync_args[@]}, ${filter_args[@]},
+# $DATEFMT,
+#
+function src_prepare
+{
+ local mirrors_dir="${src_path}/${src_mirror_reldir}"
+ local old_mirror="${mirrors_dir}/old"
+ local new_mirror="${mirrors_dir}/new"
+ local tmp_mirror="${mirrors_dir}/tmp"
+ local bakdate
+
+ assert_is_dir "$src_path" || return $?
+
+ message 'Create mirror of the source tree...'
+
+ if [[ ! -d "$mirrors_dir" ]]
+ then
+ create_dir "$mirrors_dir" || return $?
+ create_dir "$tmp_mirror" || return $?
+ ln -s 'none' "$old_mirror" || return $RET_ERROR
+ else
+ replace_symlink "$new_mirror" "$old_mirror" || return $?
+ create_dir "$tmp_mirror" || return $?
+ fi
+
+ "$RSYNC" \
+ --exclude="/${src_mirror_reldir}/***" \
+ "${rsync_args[@]}" \
+ "${filter_args[@]}" \
+ --link-dest="$src_path" \
+ "${src_path}/" "$tmp_mirror" || return $RET_ERROR
+
+ bakdate=`date +"$DATEFMT"`
+ mv "$tmp_mirror" "${mirrors_dir}/${bakdate}" || return $RET_ERROR
+ ln -s "$bakdate" "$new_mirror" || return $RET_ERROR
+
+ message 'Finished creating mirror of the source tree.'
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/variables b/variables
new file mode 100644
index 0000000..e3eb451
--- /dev/null
+++ b/variables
@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+#
+# variables.
+#
+# Copyright 2015 - 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+## CONSTANTS
+typeset -r CONFIG=/etc/rsync-backup/config
+
+typeset -r RET_TYPES='
+ RET_SUCCESS RET_FAILURE RET_BADSYNTAX RET_FILE_ERR RET_ERROR'
+typeset -r RET_SUCCESS=0
+typeset -r RET_FAILURE=1
+typeset -r RET_BADSYNTAX=2
+typeset -r RET_FILE_ERR=4
+typeset -r RET_ERROR=8
+
+typeset -r DATEFMT='%Y-%m-%d_%H%M%S'
+
+# Shell options:
+# nullglob: In case of a non-matching glob, return nothing.
+# dotglob: Include dotfiles (except . and ..) in glob matching.
+typeset -r SHOPTS='dotglob'
+typeset -r SHOPTS_AFTER_GET_ARGS='nullglob'
+
+# Always give these variables and functions to remotely executed commands,
+# as they are very commonly used.
+# These will not be listed as required.
+# Note that $host is given as argument to run_function.
+typeset -r REMOTE_DEFAULT_VARS="$RET_TYPES verbose debug host"
+typeset -r REMOTE_DEFAULT_FUNS='prefixed_write message debug error
+ debug_error'
+
+
+## Global program variables.
+typeset src src_path src_host
+typeset dest dest_path dest_host
+typeset -a filter_args
+typeset -i n
+
+
+## Variables that may be configured.
+typeset RSYNC=rsync
+
+typeset verbose=true
+typeset debug=true
+
+# Location to put mirrored trees on the source.
+typeset src_mirror_reldir='.backup'
+
+typeset -a rsync_base_args
+typeset -a rsync_mirror_args
+typeset -a rsync_transfer_args
+typeset -A bakpath
+typeset -A filter_file
+typeset filter_file_all # Applied after specific filter.
+
+# Base arguments for rsync (-rptogAXlHS, --timeout=, --numeric-ids, --info=).
+# These are used both for creation of the source mirror and the transfer of
+# that mirror to the destination. In both cases, attributes may be added.
+rsync_args=(
+ --recursive
+ --perms --times --owner --group
+ --acls --xattrs
+ --links
+ --hard-links
+ --sparse
+ --numeric-ids
+ --timeout=60
+ #-vv
+ --info=progress2
+ )
+
+source "$CONFIG"
+
+
+# Set the globabl variables $src, $dest_path and $filter_args according to
+# command line parameters and the configuration.
+#
+# $@: command line parameters
+#
+# requires the shell option nullglob to be _unset_. (for test -v to work)
+#
+function get_args
+{
+ if [ $# -eq 0 ]
+ then
+ message "Usage: $exec_name <src> [<dest>]" 2>&1
+ return $RET_BADSYNTAX
+ fi
+
+ local l_src="$1"
+ local l_dest
+
+ # Set $src_path, $src_host and $src.
+ resolve_host_path src "$l_src" || return $?
+
+ if [ $# -gt 1 ]
+ then
+ l_dest="$2"
+ elif [[ -v bakpath[$src] ]] # $# -eq 1
+ then
+ l_dest="${bakpath[$src]}"
+ else
+ error "Backup path for \"$src\" neither configured" \
+ "nor specified on command line."
+
+ return $RET_BADSYNTAX
+ fi
+
+ # Set $dest_path, $dest_host and $dest.
+ resolve_host_path dest "$l_dest" || return $?
+
+ filter_args=()
+ if [[ -v filter_file[$src] && -f "${filter_file[$src]}" ]]
+ then
+ filter_args+=("--filter=merge ${filter_file[$src]}")
+ fi
+ if [[ -v filter_file_all && -f "$filter_file_all" ]]
+ then
+ filter_args+=("--filter=merge ${filter_file_all}")
+ fi
+
+ return $RET_SUCCESS
+}
+
+
+# Split host:target like rsync and convert local (!) relative paths to
+# absolute ones. This is needed at least for $src_path which is used as
+# key for associative arrays.
+#
+# The results are written to the global variables $<vtype>, $<vtype>_host
+# and $<vtype>_path, where <vtype> is either src or dest. $<vtype> gets
+# set to the original to be resolved string, unless we are local, i.e.
+# there is no host.
+#
+# Note: This is not a place for bash to shine. Perl for instance would do
+# this much more easily.
+#
+# Examples:
+# - a:b -> (a, b)
+# - a/b:c -> ('', a/b:c)
+#
+# $1: which global variables to set: (src|dest)
+# $2: path string potentially containing host prefix
+#
+# sets: ${$1}_host: host name (may be empty)
+# ${$1}_path: path (absolute, if local, i.e. ${$1}_host='')
+# ${$1}: ${$1}_path, if ${$1}_host='',
+# ${$1}_host:${$1}_path (= $2), elsewise.
+#
+function resolve_host_path
+{
+ local vtype="$1"
+ local combined="$2"
+ local host path
+
+ # Split host and path.
+ host=${combined%%:*}
+ if [[ "$host" == "$combined" || "$host" =~ '/' ]]
+ then
+ host=''
+ path="$combined"
+
+ # Resolve to absolute directory.
+ # pwd -P avoids symlinks.
+ # Do not resolve symlimks in ${path} or test for its existance.
+ # `- realpath could do that.
+ if ! [[ "$path" =~ / ]]
+ then
+ path="$(pwd -P)/${path}"
+ fi
+
+ combined="$path"
+ else
+ path="${combined#*:}"
+ fi
+
+ # Set global variables.
+ if [[ "$vtype" == 'src' ]]
+ then
+ src_host="$host"
+ src_path="$path"
+ src="$combined"
+ else # "$vtype" == 'dest'
+ dest_host="$host"
+ dest_path="$path"
+ dest="$combined"
+ fi
+
+ return $RET_SUCCESS
+}
+
+# vi: ft=bash ts=2 sw=2 noet
diff --git a/write b/write
new file mode 100644
index 0000000..081bbd2
--- /dev/null
+++ b/write
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+#
+# write - output functions.
+#
+# Copyright 2019 Einhard Leichtfuß
+#
+# This file is part of rsync-backup.
+#
+# rsync-backup is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published
+# by the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# rsync-backup is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with rsync-backup. If not, see <https://www.gnu.org/licenses/>.
+#
+
+
+# Write message, prefixed by either 'local' or the respective current host.
+# Each argument is printed on its own line.
+#
+# requires: $host (unset or empty if running on localhost)
+#
+# $1: secondary prefix
+# ${@:2}: messages: string+
+#
+function prefixed_write
+{
+ local prefix
+ local msg
+
+ test -n "$host" && prefix="${host}: "
+ test -n "$1" && prefix+="${1}: "
+
+ printf '%s%s\n' "$prefix" "$2"
+
+ for msg in "${@:3}"
+ do
+ printf '%s%s\n' "${prefix//?/ }" "$msg"
+ done
+}
+
+function message
+{
+ $verbose && prefixed_write '' "$@"
+}
+
+function debug
+{
+ $debug && prefixed_write "Debug" "$@"
+}
+
+function error
+{
+ prefixed_write "Error" "$@" 2>&1
+}
+
+function debug_error
+{
+ $debug && prefixed_write "Debug" "$@" 2>&1
+}
+
+# vi: ft=bash ts=2 sw=2 noet