Table of Contents
  • Docs >
  • 40. The scratch-folder
Shortcuts

40. The scratch-folder

ShimmerCat needs a directory in a local filesystem to store cached and pre-processed versions of the files which are served. We call that directory “the scratch folder”.

The structure of this folder follows a fixed convention detailed in this document.

40.1. Default location

The scratch folder is created in the directory where ShimmerCat runs, with the name .shimmercat.loves.devs. It is possible to chose another name using the command line option --scratch-dir-name, but note that the option only affects the name of the folder, not its location. To change the folder’s location, the option --working-dir can be used, which will also affect where ShimmerCat searches for the devlove.yaml file. The corolary is that a devlove.yaml file will always be next to its scratch folder, and yes, they are expected to hold a one-to-one relation at all times.

The folder is created the first time that ShimmerCat runs, and from there on it is just re-used. The scratch folder is populated and maintaned for the most part automatically, but sometimes ShimmerCat may need a bit of help from the user.

40.2. Contents of the scratch folder

40.2.1. Certificates

The certificates for the sites configured in the devlove.yaml are stored in the scratch folder with these names:

  • cert.csr: this is a certificate signing request that can be submitted to a certification authority (CA) to get a signed, real certificate. It already includes all the domains configured in the devlove.yaml file. Notice that this csr file is aimed for simple domain validation.

  • tweaks.yaml: this file contains global settings for ShimmerCat. This file is automatically generated.

  • cert.pem is the actual certificate used to serve all the domains in the devlove.yaml file. It should thus cover all those domains.

  • privkey.pem is a RSA private key for a cert.pem. This file is generated by ShimmerCat together with cert.pem because some tools have use for the private key in this format, but ShimmerCat itself doesn’t use this file.

  • privkey.unencrypted-pkcs8.pem is the private key for cert.pem as an unencrypted PCKS#8 file, this is the private key that ShimmerCat uses for cert.pem above.

  • sni-certs/<domain>/... locations for SNI certificates. For each <domain>, the individual files should have names as in the general case: cert.pem and privkey.unencrypted-pkcs8.pem. As of this writing, ShimmerCat QS doesn’t create SNI certificates automatically. To support LetsEncrypt alpn-01 challenges, files with names le_alpn_01_cert.pem and le_alpn_01_privkey.unencrypted-pkcs8.pem are used as certificate and private key respectively for TLS connections that announce acme-tls/1 as the only supported protocol via ALPN.

  • acme/... locations for ACME HTTP challenge files. A file with path acme/.well-known/acme-challenge/abcd in the scratch folder is available at the URL /.well-known/acme-challenge/abcd, but only in the port for unencrypted, non-TLS HTTP; the one specified with the --http2https option.

If the files above are not present, ShimmerCat populates them with development certificates.

40.2.2. Assets cache

The folder r-cache inside ShimmerCat’s scratch folder is used for storing processed assets. For example, copies of CSS, JS and HTML files compressed with GZip and Brotli are stored there, as well as the SHA hash needed to create their etags.

For images, the cache also stores the progressive or interlaced breakpoints used by. When using content-disposition: use-json in a view, the cache also stores the compiled templates.

This folder is created automatically, and its contents are updated transparently every time you update the files of your site inside a root-dir folder.

However, it is a good idea to clear this folder between major upgrades of ShimmerCat. It may also be needed if a resource there becomes corrupt.

To clear the assets cache, stop ShimmerCat if it is running, delete redis-dump.rdb and the r-cache folder.

40.2.3. Dynamic linker cache

ShimmerCat does JIT-compiling for certain data-driven functions, in particular string search. After those functions are compiled, the compiled version is stored in the t-cache folder inside ShimmerCat’s scratch folder.

40.2.4. Redis files

ShimmerCat runs an instance of Redis internally (or connects to an externally running redis instance), and therefore a few files required by Redis are also stored in the scratch folder:

  • redis-dump.rdb: a dump of the Redis database.

  • redis.log: the Redis log.

  • redis.sock: a Unix domain socket where Redis listens.

  • redis.pid: the pid of the Redis process. If for some reason ShimmerCat is terminated before it can shutdown the Redis instance, ShimmerCat will use both redis.pid and redis.sock to try to connect to the same Redis instance.

ShimmerCat uses database 0 inside the Redis instance, and keys are of reasonable length and unlikely to collide with other applications or with other ShimmerCat instances serving different sites.

40.2.5. Read-only databases

Directories and files in the scratch folder, under rdonly-dbs, are used to hold or link read-only databases. At the moment, that includes the files for the IP-records functionality, which must be copied or linked to rdonly-dbs/ip-records/. Note that the file name is used as the key of the ip_records table in the change-headers-in and change-headers-out functions.