19. Fetch backends

ShimmerCat uses a simple interface for pulling static assets. At the moment, we have three implementations of that interface:

• One for simple files lying around in a filesystem, filesystem

• One for fetching files through HTTP/1.1, without HTTPS, http-fetch

• Another for fetching files through HTTP/1.1 through the same consultant that the electric domain uses, use-consultant.

fetch backend	Can HTTPS?	Can use pooling?	Value at root-dir
filesystem	of course not	not needed	string with path
http-fetch	no	yes	(1)
use-consultant	yes	yes	(2)

Notes:

(1) The http-fetch backend is enabled by having an object under root-dir with a fetch-backend keyword with a value http, and then a few other pieces as explained in HTTP 1.1 for statics.

(2) The use-consultant backend is enabled by having an object under root-dir with a keyword use-consultant.

19.1. A brief note on terminology

Notice the spelling, “pulling”, as in tying a rope to a tire and dragging the tire from one place to the other by pulling the rope.

In some parts of the documentation we also use the term “pooling” connected to static assets. That one has a different spelling, and it comes from the word “pool”. Think about a swimming pool, which instead of water is full of HTTP connections that ShimmerCat can use and re-use to fetch assets.

Once you understand that there are different fetch backends, you can configure the server to pull resources using HTTP/1.1. There are two different ways to go about this, using different fetch backends.

19.2. The http-fetch fetch backend

This backend supports pooling, and specifying both a host and a path-prefix. HTTPS is not supported

To use it, instead of having root-dir in the devlove contain a string, it should contain the backend configuration. Like this:

---
shimmercat-devlove:
  domains:
    elec development.aspdotnetstorefront.outdoorexperten.se:
      root-dir:
        fetch-backend: http
        http-port: lookup(development.aspdotnetstorefront.outdoorexperten.se):80
        use-host: development.aspdotnetstorefront.outdoorexperten.se
      views-dir: views-dir
      changelist-settings:
        tOld: 7200
        tNew: 20

Notice there:

• views-dir must be given if we are using any backend which is not a local-filesystem backend.

• The same as in other cases, it’s important to set the changelist-settings

• It’s also possible to specify a key path-prefix at the same level than fetch-backend with a fragment to prepend to the path.

19.3. The use-consultant fetch backend

This uses one of the consultants configured for this electric domain, the one that you name it. As such, it supports HTTPS and pooling, although such configuration is just taken from the consultant. To use it, have an object under root-dir with a keyword use-consultant and the name of the consultant to use (which you should have also defined):

shimmercat-devlove:
   domains:
      elec www.supershoes.com:
          consultant:
            connect-to: "8080"
          root-dir:
            use-consultant: default

20. Dynamic vs static assets

ShimmerCat separates dynamic and static assets in the following ways:

• Dynamic assets are never cached nor considered for inclusion in acceleration rules.

• Static assets are cached always, and their file path is used as the main component of the key in the cache. Any query string components in the URL are ignored. Note that ShimmerCat uses changelists to detect when one of these files changes. Furthermore, ShimmerCat takes care of creating alternate representations, e.g. different compression formats or image re-encodings.

• Static assets are considered for inclusion in accelerator rules, including automatic push lists.

• Dynamic assets are handled with a lot of timeout lenience, meaning that if the backend application needs to process a request that takes a few minutes, ShimmerCat will make reasonable efforts to let the request live that long.

• Static assets are expected to be there, and thus when fetched via FastCGI or HTTP(S), ShimmerCat will only wait for a short period of time before closing the connection and declaring the asset missing.

The exception to the rules above is “generated” assets, whose syntax is explained in more detail in the chapter about the rewrite engine. The following applies to them:

• Generated assets require a view, and they are fetched using a consultant, possibly designated in their view.

• Generated assets are cached, just like ordinary static assets. Their lifetime in the cache follows the same rules than static assets, except for the fact that they can’t be designated in a changelist.

• Generated assets are not expected to be files. Therefore, their mime type is deduced using the Content-Type header from the backend, and their cached entry is computed using their original URL-path and any query string, before URL-rewrites.

• Generated assets are expected to be available in just a few seconds, if the application’s endpoint takes longer than that to produce them, ShimmerCat will declare the asset missing.