Skip to content

Setup sc_pack without ansible

For simplicity, we will assume that you have created a in your home, /home/shimmercat/test/shimmercat-accelerator, where you want to install sc_pack. We will call this the install_dir.

1. Installation

  • Move to the install_dir directory, for example via command line:
$ cd /home/shimmercat/test/shimmercat-accelerator
  • Download the latest sc_pack to that directory. You can either find the download link by visiting the API page, or by directly using the below download command (requires jq):
curl -O $(curl -s  https://accelerator.shimmercat.com/presentation-api/v1/download/sc_pack/?format=json | jq -r .download_url)
  • Install dependencies:
$ sudo apt-get install python3 python3-dev virtualenv git
  • Create a Python 3 virtualenv:
$ virtualenv venv -p python3
  • Activate the virtualenv:
$ source venv/bin/activate
  • Install sc_pack on your virtualenv:
$ pip install /home/shimmercat/test/shimmercat-accelerator/<sc_pack-version>.whl
  • To verify that sc_pack is properly installed, run:
$ sc_pack -h

2. Directory hierarchy and configuration

The programs need a directory as the top of the hierarchy, and some information for configuration. This information contains details about your setup, and secrets needed for the programs to send and receive data from our cloud service.

The configuration file

All configuration is structured in a YAML file, called sc_pack.conf.yaml, which has nesting levels. That's what we Take a look to prepare the directory hierarchy to know more details about what each field means on the sc_pack.conf.yaml. We have created an example file below - remember to edit the fields according to your details.

---
# This is a YAML document with the configuration items
# we need to set up the server with sc_pack.
# For the format, see example below.
#
# All the directories defined are relative to install_dir
shimmercat:
    root_dir: www
    devlove_file_path: devlove.yaml
    scratch_dir_name: shimmercat-scratch-folder
    listen_port: "4043"
    http2https_port: "8084"
    bots_views_dir: views-dir
# Update this value with the value of your authentication token
api_access_token: <your_authentication_token>
user: shimmercat
user_group: shimmercat
# Update this path with the path you want the sc_pack files to be at
install_dir: /home/shimmercat/test/shimmercat-accelerator
logs_host_name: localhost
# Update this value with the value of your deployment site's `name16` you will create on the next step
deployment_site_long_name: <your_name16>
# Update this value with the value of your deployment site's `secret16` you will create on the next step
deployment_site_long_secret: <your_secret16>
amqp_server_url: amqp.staging.c.shimmercat.com
amqp_vhost: skyloft
enable_bots_blocking: True
humanity_validator_host: localhost
humanity_validator_port: 8080
google_recaptcha_site_key: 6LfimFIUAAAAAACMycLvA3wetRR9v6ejSbf7uu2v
google_recaptcha_site_secret: 6LfimFIUAAAAAEpCe1QNHwr2p46AMKbUoOBdLWbE
transit_encryption_key: test-accelerator.shimmercat.com.no_enc
deployment_group: test
dht_bind_port: 48101
dht_bootstrap_node: dht.shimmercat.com:26131

If you don't have a user called shimmercat you can either create it, or use a different one. In any case, please be sure to update the user and user_group above accordingly.

  • Copy the content above to /home/shimmercat/test/shimmercat-accelerator/sc_pack.conf.yaml

You can also get a similar copy of the config file by using the sc_pack command:

$ sc_pack extract example.sc_pack.conf.yaml

Directory hierarchy

Once the configuration file is created, it's time to setup the directory hierarchy. If you have already a file called sc_pack.conf.yaml in the directory from where you are running that command, run:

    $ sudo sc_pack create

Otherwise you can let sc_pack create the deployment environment by typing:

$ sc_pack create -f /home/shimmercat/test/shimmercat-accelerator/sc_pack.conf.yaml

Subject to the same rule, it's possible to omit the -f in many sc_pack sub-commands. E.g, if the current directory is /home/shimmercat/test/shimmercat-accelerator and the file /home/shimmercat/test/shimmercat-accelerator/sc_pack.conf.yaml exists and you run sc_pack create, the configuration file taken is /home/shimmercat/test/shimmercat-accelerator/sc_pack.conf.yaml.

  • Check that the directories are properly created:
$ ls
$ celery redis shimmercat   devlove.yaml sc_logs_agent shimmercat-scratch-folder views-dir  # And some others
  • Start the accelerator:
$ sc_pack supervisord

You will see some logs similar to:

2019-05-18 14:52:00,385 INFO success: redis entered RUNNING state,
process has stayed up for > than 10 seconds (startsecs)

2019-05-18 14:52:00,385 INFO success: shimmercat entered RUNNING state,
process has stayed up for > than 10 seconds (startsecs)

2019-05-18 14:52:00,385 INFO success: kv_store entered RUNNING state,
process has stayed up for > than 10 seconds (startsecs)

2019-05-18 14:52:00,385 INFO success: humanity_validator entered RUNNING state,
process has stayed up for > than 10 seconds (startsecs)

To start the accelerator at boot time, see here.

Inside every service directory there is a directory named data, where the logs for each service are stored. If there are errors, please check the logs for more details. For example, if you see the error celery (exit status 1; not expected), you can check more details in the log file /home/shimmercat/test/shimmercat-accelerator/celery/data/celery.log.

Note that it's expected that the celery and the sc_logs_agent don't start with the current configuration. The reason is that we are still missing the info for deployment_site_long_name and deployment_site_long_secret that are needed to let Celery authenticate with Celery broker.