45. Configuration reference¶
45.1. The devlove.yaml
file¶
ShimmerCat QS uses two instance-wide configuration files, both of which are expected to be at fixed locations relative to the server’s working directory:
devlove.yaml
contains basic configuration about which domains to serve.<scratch-folder>/tweaks.yaml
contains technical settings that can be changed to balance security and features.
The basic features of this file are covered in detail in our tutorials in this documentation. Here is the full reference for the file, using json-schema, though we have used YAML instead of JSON for the schema so that it is easier to read. Scroll to after the text-box to find additional information about each object.
---
title: ShimmerCatDevlove
$schema: http://json-schema.org/draft-04/schema#
type: object
properties:
shimmercat-devlove:
type: object
properties:
domains:
type: object
minProperties: 1
patternProperties:
"elec +[a-zA-Z0-9.-]+":
$ref: "#/definitions/electricDomain"
"api +[a-zA-Z0-9.-]+":
$ref: "#/definitions/apiDomain"
additionalProperties: false
required: ["domains"]
additionalProperties: false
required: ["shimmercat-devlove"]
definitions:
electricDomain:
type: object
title: ElectricDomain
properties:
root-dir:
oneOf:
- type: string
- $ref: "#/definitions/rootDirUseConsultant"
- $ref: "#/definitions/rootDirFetchBackend"
views-dir:
type: string
consultant:
$ref: "#/definitions/consultant"
consultants:
type: object
additionalProperties:
$ref: "#/definitions/consultant"
cache-key:
type: string
change-url:
type: array
items:
- type: string
changelist-settings:
type: object
properties:
tNew:
type: number
tOld:
type: number
required: ["tNew", "tOld"]
prob-accelerator-kicks-in:
type: number
bot-protection-enabled:
type: boolean
required:
- "root-dir"
additionalProperties: false
apiDomain:
type: object
title: ApiDomain
properties:
port:
$ref: "#/definitions/networkAddress"
required:
- port
applicationPort:
type: object
title: ApplicationPort
properties:
# TO-DO: We can make the `connect-to` property more precise
"connect-to":
$ref: "#/definitions/networkAddress"
"document-root":
type: string
"application-protocol":
type: string
pattern: |
fastcgi|fcgi|http|http11|HTTP/1\.1|HTTP
"encryption-level":
type: string
pattern: |
plain|tls
"timeouts":
$ref: "#/definitions/applicationLifecycleTimeouts"
"use-host":
type: string
"forward-error-pages":
type: boolean
"enable-ip-records":
type: boolean
"change-headers-in":
type: boolean
"change-headers-out":
type: boolean
required: ["connect-to"]
networkAddress:
type: string
pattern: "^(((lookup\\([a-zA-Z0-9.-]+\\))|([0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+))?:)?([0-9]+)$"
applicationLifecycleTimeouts:
type: object
title: Application Lifecycle Timeouts
properties:
# Will add more, like "connection" and "handshake" later
"inactivity":
type: "number"
"connection":
type: "number"
"handshake":
type: "number"
consultants:
type: object
additionalProperties:
$ref: "#/definitions/applicationPort"
consultant:
oneOf:
- $ref: "#/definitions/networkAddress"
- $ref: "#/definitions/applicationPort"
rootDirUseConsultant:
type: object
properties:
"use-consultant":
type: string
additionalProperties: false
required:
- "use-consultant"
rootDirFetchBackend:
type: object
properties:
"fetch-backend":
type: string
"http-port":
$ref: "#/definitions/networkAddress"
"use-host":
type: string
"concurrency-limit":
type: number
"path-prefix":
type: string
required:
- "fetch-backend"
- "use-host"
The following sub-sections explain the objects defined in the schema above.
45.1.1. Top objects¶
The top object in the devlove.yaml
should only contain one property, shimmercat-devlove
,
and right below it another property domains
:
---
shimmercat-devlove:
domains:
...
domains
should contain an object, with one property for each domain served by ShimmerCat.
45.1.2. Domains¶
The contents of the domains
object are two types of domains: file-serving domains, which
we call electric, and pure-forwarding domains, which we call api domains (using lowercase
for “api”).
Electric domains can be recognized automatically by the subkeys root-dir
or consultant
.
Like this:
shimmercat-devlove:
domains:
www.mysite.com:
root-dir: www
They can also be distinguished by prefixing the word elec
to the key with the domain
name.
API domains can be recognized automatically by specifying the subkey port
with the web
application port to which ShimmerCat should proxy requests, or by prefixing the word api
to the key with the domain name, like this:
shimmercat-devlove:
domains:
api my-api.mysite.com:
port: "8080"
Note that the value of the port
property should follow the network-address schema,
described below.
Of course, you can mix and match domains of different types in a single devlove.yaml
file.
Like this:
shimmercat-devlove:
domains:
api1.mysite.com:
port: "8080"
www.mysite.com:
root-dir: www
What follow are all properties of the electric domains.
45.1.3. The root-dir¶
This is a string denoting a directory with static assets to serve on the domain.
Or an object, describing an equivalent mean to obtain the assets.
ShimmerCat QS, being a caching server, will just read the files
from this place as they are requested by website visitors and copy-and-process them to
its internal cache in the scratch folder.
This means that you can mount a high latency filesystem and use it as your root-dir
, and
still obtain a very fast website.
This is the only mandatory property in an electric domain configuration object.
45.1.4. The views-dir¶
This is where views are expected to be.
If left unspecified and root-dir
is a string, then the root-dir
will be used.
Note that view files are expected to be in a local filesystem, since at best
they are just cached for a few seconds.
45.1.5. Consultant(s)¶
In ShimmerCat lingo, a consultant is an external application that provides either contents or base data to create contents (depending on the content-disposition). In the first case “consultant” is synonymous to “origin” and “application backend”.
A single domain can access more than one consultant, via the consultants
property
of the domain configuration object.
For example:
...
consultants:
my-php-app:
...
my-node-js-app:
...
The snippet above defines two consultants with names my-php-app
and my-node-js-app
, both of
which can be served in the same domain.
Their names can be used in a view to decide where to forward a request to.
Whenever a consultant is needed but no consultant name is provided, ShimmerCat QS fills in the
missing name with default
, and tries to use that one.
The property consultant
, the singular of consultants
, is just a shortcut to
consultants:
default:
...
Consultants configuration also includes the following properties:
document-root
: passed asDOCUMENT_ROOT
to FastCGI applicationconnect-to
: the application port where to connect to the service. It follows the standard network port syntax. This is the only required property.application-protocol
: one offastcgi
orhttp
, to denote FastCGI or HTTP 1.1. This is the expected protocol to talk to the backend. When not specified, ShimmerCat tries to guess.encryption-level
: eitherplain
ortls
. ShimmerCat usesplain
by default.timeouts
: different timeouts for contacting the application backend.use-host
: Optional. If specified, it should be a string to use as the value of theHost
HTTP 1.1 header; also used for theHTTP_HOST
FastCGI field. Useful in those cases when one needs to replace the host header in outgoing requests. Otherwise outgoing requests go with the sameHost
header value that came in the original request.
45.2. Command Line Reference¶
45.2.1. Command line options across modes¶
ShimmerCat, as of version 1.5.0 has two public-facing modes of operation: devlove and Internet The modes have slightly different options, and some of them have different defaults across modes.
devlove
: a handy option to run while developing. It enables SOCKS5 by default, relaxes security settings and strives to rely as much information as possible to web developers.internet
: an option to run when facing the Internet. It disables SOCKS5 by default, tightens security settings and strives for reliability and stability. In this mode ShimmerCat runs a master process and a number of workers.
In addition, there are two non-public modes:
worker
: this mode is used automatically by the worker processes spawned ininternet
mode.changelist-daemon
: used to keep the cache fresh.
In any of the modes, you can see which options ara available via the --help
sub-option:
$ shimmercat devlove --help
A mode for web developers
...
45.2.2. General options¶
--listen
,-l
: Network address to bind to. It has several supported formats. If given as an integerport-no
, it will bind to that port at the localhost interface127.0.0.1
(not to0.0.0.0
). It can also be given as a pairip-number:port-no
, in which case it will bind to that ip address and port, or asunix://<abs-or-rel-path
, in which case the server will bind to a local Unix socket. This last choice may come handy for different proxy-ing setups.--socks5-listen
,-5
: Port where to bind the SOCKS5 proxy. Accepts the same formats than the--listen
option.--s5-forward
,-w
: Instructs the SOCKS5 built-in proxy to forward connections which are not handled by local domains. For example, if yourdevlove.yaml
file specifies thatwww.mysite.com
should be served locally, but you have URLs referencing an external site likefonts.google.com
, this option will “pass-through” connections to the later domain. Without this option, those connections are blocked.--scratch-dir-name
: The name of the scratch directory. By default, it is “.shimmercat.loves.devs”. The scratch directory contains the Redis database that ShimmerCat uses for internal comunications and caching, the local filesystem socket to Redis and the cache of representations.--worker-count
: How many workers to span. This option is available only ininternet
mode.--disable-socks-5
: Disable SOCKS5. Use this option for enhanced security if you are abusingdevlove
mode to serve an actual website. We recommend you to upgrade instead.--enable-socks-5
: Use this option to enable SOCKS5 ininternet
mode.--wait-for-redis-to-start
: Time in milliseconds to wait for Redis to start. By default, ShimmerCat makes a short pause of 50 milliseconds before trying to connect to Redis. That is enough in most modern hardware, but in old hardware or in otherwise throttled down CPUs Redis may take longer to start. Use this option in those cases.--working-dir
: Directory that ShimmerCat should consider as current. This is entirely equivalent to starting ShimmerCat at that directory.--log-codes
: Command line option used to specify what to log. For example,--log-codes=all\\connection
can be used to enable all the logs minus logs related to connections. For more details, read the page on logging.--http2https
,-t
: Gives ShimmerCat a port to make HTTP to HTTPS redirects. For details, see the corresponding section in the Internet mode article.
45.2.3. Options for development¶
The following options are handy to debug protocol-specific issues:
--disable-http-11
: Disable HTTP/1.1. That is, only accept HTTP/2 connections.--disable-http-2
: Disable HTTP/2, that is, only accept HTTP/1.1 connections.
When developing a website, it is handy at times to disable all cache directives or even HTTP/2 Push. These two options are for that:
--disable-ua-caching
: Changes the cache headers to ask the browser not to cache any requests, for all requests in all virtual domains ran by the server. In particular, this removes theetag
header and changes thecache-control
header tono-cache, no-store, max-age=0, private
.--disable-push
: Disables HTTP/2 Push in all virtual domains ran by the server. Notice that that otherwise ShimmerCat respects the connection settings indicated by the client as part of a SETTINGS frame.--show-proxied-headers
: Shows the headers which are forwarded to an inner application, and the headers which are received by that application.--sim-bandwidth
: Part of the network simulator. Limits server bandwidth for a single HTTP/2 connection to the number given here. If you don’t specify a unit, the server will assume bytes per second as the unit. Units accepted are a subset of the units described at Wikipedia’s page:kbit/s
,kb/s
(kilobits per second),Mb/s
,Mbit/s
(megabits per second),Kibit/s
(kibits per second),Mibit/s
(mibits per second),kB/s
(kylobytes per second) andMB/s
(megabytes per second). Example:--sim-bandwidth=100kb/s
.--sim-latency
: Part of the network simulator. Adds latency to packets in their way out, by delaying them by the amount of time specified in this option. If no units are specified, ShimmerCat assumes that the time is given in seconds and fractions of a second. Two units are allowed here,s
andms
. Example:--sim-latency=110ms
.