Synchronisation of files with S3 using the hash of the file contents.
Originally based on Alex Kudlick’s aws-s3-sync-by-hash.
aws s3 sync ... command only uses the time stamp of files to decide what files need to be copied. This utility looks at the md5 hash of the file contents.
thorp Usage: thorp [options] -V, --version Display the version and quit -B, --batch Enabled batch-mode -s, --source <value> Source directory to sync to S3 -b, --bucket <value> S3 bucket name -p, --prefix <value> Prefix within the S3 Bucket -P, --parallel <value> Maximum parallel upload/copy operations -i, --include <value> Include matching paths -x, --exclude <value> Exclude matching paths -d, --debug Enable debug logging --no-global Ignore global configuration --no-user Ignore user configuration
If you don’t provide a
source the current diretory will be used.
--exclude parameters can be used more than once.
--source parameter can be used more than once, in which case, all files in all sources will be consolidated into the same bucket/prefix.
Batch mode disable the ANSI console display and logs simple messages that can be written to a file.
Configuration will be read from these files:
- User: ~
Command line arguments override those in Source, which override those in User, which override those Global, which override any built-in config.
When there is more than one source, only the first “.thorp.conf” file found will be used.
Built-in config consists of using the current working directory as the
exclude are cumulative across all configuration files.
When considering a local file, the following table governs what should happen:
|#||local file||remote key||hash of same key||hash of other keys||action|
|2||exists||is missing||-||matches||copy from other key|
|3||exists||is missing||-||no matches||upload|
|4||exists||exists||no match||matches||copy from other key|
|5||exists||exists||no match||no matches||upload|
To build as an executable jar, perform `sbt assembly`
This will create the file `cli/target/scala-2.12/thorp`
Copy this file to your `PATH`.