To manage the edge servers running ShimmerCat Accelerator, we install a “pack” of programs called
sc_pack. The installation consists of two steps: first you need to install
sc_pack itself, and then you need to ask
sc_pack to prepare the hierarchy of directories.
3.1. 1. Installation¶
First, you will need to install python3, python3-dev, virtualenv and git on the server. You can install them by executing:
$ sudo apt-get install python3 python3-dev virtualenv git
We highly recommend that you use a Python 3 virtualenv. This virtualenv will contain an isolated copy of Python 3,
sc_pack, supervisord and all the libraries needed for the tools and for the Celery worker. Since it’s so tied to
sc_pack, we recommend that you create the virtualenv as a subdirectory in the hierarchy we mentioned before, the so called
<venv> directory. This can be done by executing:
$ virtualenv /path/to/install_dir/venv -p python3
Don’t forget to replace
/path/to/install_dir/ with the actual path you use. Then you can activate it:
$ source /path/to/install_dir/venv/bin/activate
curl -O $(curl -s https://accelerator.shimmercat.com/presentation-api/v1/download/sc_pack/?format=json | jq -r .download_url)
After you have downloaded
sc_pack to your server, with the venv activated, do:
$ pip install /path/to/<sc_pack-version>.whl
sc_pack should be installed. To verify that it is properly installed run:
$ sc_pack -h
3.2. 2. Prepare the configuration file and the directory hierarchy¶
The programs need a directory as the top of the hierarchy, and some information for configuration. This information contains details about your setup, and secrets needed for the programs to send and receive data from our cloud service.
3.2.1. The configuration file¶
All configuration is structured in a YAML file, which has nesting levels. In the rest of this documentation, we sometimes refer to a nested field by using a
/, for example
shimmercat/root-dir refers to the field
root-dir under the object
There is an example file inside the
sc_pack redistributable, that can be obtained by executing:
$ sc_pack extract example.sc_pack.conf.yaml
Copy the content that appears in the terminal and create a file with extension
In the configuration file, you need to edit the following:
install_dir: the top directory for the hierarchy of files.
user: the user that will be used to run all the programs in
sc_pack. This can be
root, although is not needed nor recommended. Instead, we recommend you create a special user for this. Said user needs all access to the hierarchy of files, but to nothing outside of it. If you want, you can use the command
adduser --system --disabled-login --no-create-home --group sc_packto create a user called
sc_packand an associated group.
shimmercat/port: the port where ShimmerCat will listen for web requests.
shimmercat/root-dir: this setting is used to put an initial folder in the
devlove.yamlfile for where to serve files from. By default it’s inside the hierarchy and called
www, but you can also write an absolute directory here. Just be sure the user you provided before has read access to this directory.
shimmercat/bots-views-dir: this directory contains ShimmerCat views for the blocking bots feature. It is only required when enable_bots_blocking is True, and there is not a views-dir defined in devlove.yaml. If there is a views-dir in devlove.yaml, it is used by default.
logs_host_name: a hostname to identify your logs. Any hostname will do, so you can either write the output of the
hostnameprogram, or some other label that you see fit. This hostname will be used later in the dashboard with our service to identify your deployment.
enable_bots_blocking: <True/False> will enable the bots blocking mechanism.
humanity_validator_host: host where the service to enable the humanity validator runs. We need it because when an IP address has been automatically detected as a prospector bot, we ensure that it is actually a bot with a mechanism using Google reCAPTCHA. That way a user can still access the website after defeating the CAPTCHA challenge, if an IP address has been wrongly classified as bot. Note that this host should be set in the devlove.yaml consultants too.
humanity_validator_port: port that will use the service to enable the humanity validator. Note that this host should also be set in the devlove.yaml.
google_recaptcha_site_key: The Google reCAPTCHA mechanism is used to validate human visitors vs bots. You can create the key through Google reCAPTCHA admin.
google_recaptcha_site_secret: As mentioned before we use the Google reCAPTCHA mechanism to validate human visitors vs bots. You can create the secret through Google reCAPTCHA admin.
transit_encryption_key: a free-form password that sc_pack uses to encrypt files to synchronized across deployments of the same deployment group. Among other things, we use it to synchronize the certificates and their private keys. Make sure to set the same
transit_encryption_keyto all the deployments you would like to synchronize. Also, keep this key safe, as access to it could allow a third party to gain access to your certificates and their private keys. The optimization cloud service does not store this key.
deployment_site_long_name: the name16 provided by ShimmerCat cloud service
deployment_site_long_secret: the secret16 provided by ShimmerCat cloud service
deployment_group: an arbitrary name for a deployment group. A deployment group is a set of deployments that share the same set of domains, and thus are expected to be in sync regarding the
devlove.yaml, the certificates and the views. We suggest you strive to keep this name globally unique and hard to guess, as in
3.2.2. Directory hierarchy¶
Once the configuration file is created, it’s time to setup the directory hierarchy. If you have already a file called
sc_pack.conf.yaml in the directory from where you are running that command, run:
$ sudo sc_pack create
Otherwise you should run:
$ sudo sc_pack create -f configuration/file/path/sc_pack.conf.yaml
You can get a copy of the config template content that sc_pack has by executing:
$ sc_pack extract example.sc_pack.conf.yaml
You can then copy and paste that content on a file called
sc_pack.conf.yaml and then run the
create command without arguments. Subject to the same rule, it’s possible to omit the
-f in many
E.g, if the current dir is
/srv/www.example.com and the file
/srv/www.example.com/sc_pack.conf.yaml, and you run
sc_pack create, the configuration file taken is
We recommend running the command above with root privileges so that
shimmercat can be authorized to bind to port 443, without running ShimmerCat itself as root.
sc_pack ... create command also creates an example
devlove.yaml, which is ShimmerCat’s configuration file with domains and such. The format of that file is documented elsewhere, but this is a good moment to either edit it by hand or to request our cloud service to send you a version.