Configuration

Overview

The global configuration for webchanges contains basic settings for the generic behavior of webchanges, including its reports and reporters. It is written in YAML format, is called config.yaml, and is located in the following directory:

Linux: ~/.config/webchanges
MacOS: ~/Library/Application Support/webchanges
Windows: %USERPROFILE%/Documents/webchanges (the webchanges folder within your Documents folder)

It can be edited with any text editor or with:

webchanges --edit-config

Tip

(Linux) If you use this command and get an error, set your $EDITOR (or $VISUAL) environment variable in your shell with a command such as export EDITOR=nano.

For a summary of the YAML syntax, see here.

Keys starting with an underscore (_) are ignored and are used in webchanges for writing remarks in the configuration file:

_note: This is a remark

Tip

Credentials and other secrets (e.g. a reporter’s token or password) can be kept out of the configuration file by storing them in environment variables and inserting them with the !env tag; see Using environment variables.

Changed in version 3.11: Keys starting with an underscore (_) are ignored.

Changed in version 3.36: A JSON Schema config.schema.json is now installed in the same directory as --config to enable editors to provide autocompletion and validation.

Display

In addition to always reporting changes (which is the whole point of the program), webchanges by default reports newly-added (new) pages and errors (error).

You can change what is reported in the display section (named from when that’s all the program did) of the configuration:

display:
  new: true
  error: true
  unchanged: false
  empty-diff: true  # deprecated

If you set unchanged to true, webchanges will always report all pages that are checked but have not changed.

empty-diff is deprecated, and controls what happens if a page is changed but the notification is reduced to an empty string e.g. by a diff_filter. If set to true, webchanges: will report an (empty) change. If set to false, the change will not be included in the report. Use the job directive additions_only instead for similar results.

Reports and Reporters

Any report- or reporter-specific configuration must be inside the report directive in the configuration.

text, html and markdown are report types, and their settings apply to all reporters that use that type of report (for example, the stdout reporter uses text, while the email reporter with html: true uses html; see reporters for details).

Here is an example configuration that reports using UTC timezone on standard output in color, as well as HTML email (one report for each job) using an SMTP server:

report:
  tz: Etc/UTC
  text:
    details: true
    footer: true
    line_length: 75
    minimal: false
    separate: false
  html:
    diff: unified  # Deprecated; specify a :ref:`differs <differs>` in the job
    separate: true
  email:  # This is the email reporter
    enabled: true
    from: 'Web watcher <webwatcher@example.com>'
    html: true
    method: smtp
    smtp:
      host: smtp.example.com
      port: 587
      starttls: true
      auth: true
      user: 'username_goes_here'
      insecure_password: 'password_goes_here'
      utf_8: true
    subject: '{count} changes: {jobs}'
    to: 'User <user@example.com>'
    stdout:
      color: true  # value can also be bright (same as true) or normal
      enabled: true
  markdown:
    minimal: false
    show_details: true
    show_footer: true
    separate: false

Configuration options for reports is described in reports.

Configuration options for reporters is described in reporters.

Configuration options for email is described in email.

Reporters are implemented in a hierarchy, and configuration settings of a report apply to all descendant reporters:

html
└───browser
text
├───xmpp
├───email
├───mailgun
├───stdout
├───run_command
│   └───shell
├───ifttt
├───webservice (no settings)
│   ├───pushover
│   └───pushbullet
├───webhook
│   ├───slack
│   └───mattermost
├───discord
├───prowl
└───ntfy
markdown
├───telegram
├───matrix
├───gotify
└───github_issue

Note

Setting the email reporter’s html option to true will cause it to inherit from the html configuration.

Job defaults

If you want to apply or change default settings for all your jobs, add them to the job_defaults section in your config file. Under it, each sub-directive names a job kind, and holds the job directives that apply by default to the jobs of that kind, written exactly as they would be in a job. The following example will set default headers for all url jobs without use_browser:

job_defaults:
  all_jobs:
    _note: Default directives that are applied to all job kinds.
    suppress_repeated_errors: true
  url_jobs:
    _note: These are defaults for URL jobs without 'use_browser'.
    headers:
      Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
      Accept-Language: en-US,en;q=0.9
      DNT: 1
      Priority: u=0, i
      Sec-CH-UA: '"Not A(Brand";v="8", "Chromium";v="132", "Google Chrome";v="132"'
      Sec-CH-UA-Mobile: ?0
      Sec-CH-UA-Platform: '"Windows"'
      Sec-Fetch-Dest: document
      Sec-Fetch-Mode: navigate
      Sec-Fetch-Site: none
      Sec-Fetch-User: ?1
      Sec-GCP: 1
      Upgrade-Insecure-Requests: 1
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36

The above config file sets all jobs to use the suppress_repeated_errors option and url jobs without the use_browser directive to use the specified headers, as if each of these jobs contained the same headers directive, e.g.:

url: https://example.com/
headers:
  Accept-Language: en-US,en;q=0.9
  # (...)

The possible sub-directives to job_defaults are:

all_jobs: Applies to all your jobs, including those in hooks.py;
url_jobs: Applies only to jobs with the directive url without use_browser;
browser_jobs: Applies only to jobs with the directives url and use_browser: true;
command_jobs: Applies only to jobs with the directive command.

See jobs for an explanation of the different job kinds and their directives.

Handling of duplicate directives

A directive specified in a job overrides the same directive specified in job_defaults, and one specified in url_jobs, browser_jobs or command_jobs overrides the same one specified in all_jobs.

Directives whose value is a set of key-value pairs, such as headers and cookies, are instead merged one key at a time, with the same precedence: each header is taken from the job if the job specifies it, otherwise from url_jobs (or browser_jobs) if specified there, otherwise from all_jobs. Header names are matched case-insensitively. For example, with this configuration:

job_defaults:
  all_jobs:
    headers:
      Accept-Language: en-US,en;q=0.9
      User-Agent: Mozilla/5.0
  url_jobs:
    headers:
      DNT: 1
      User-Agent: Chrome

this job:

url: https://lynx.invisible-island.net
headers:
  user-agent: Lynx

is sent the headers Accept-Language: en-US,en;q=0.9, DNT: 1 and user-agent: Lynx.

To remove a default header or cookie, set it to null (no value): it is then not sent, and no less specific default can supply it. This works both in a job, to remove a header or cookie set in job_defaults, and in url_jobs or browser_jobs, to remove one set in all_jobs. Continuing the example above, this job is sent only the header Accept-Language: en-US,en;q=0.9 and, as no User-Agent header is left, webchanges’ own default one:

url: https://lynx.invisible-island.net
headers:
  DNT:
  User-Agent:

Directives whose value is a list, such as filters, are not merged: the one specified in the job replaces the default one entirely.

Changed in version 3.38: A header or cookie set to null removes the default one; previously, it was sent with the value None.

Previous names

These sub-directives were previously named all, url, browser and command (the latter originally shell), as in urlwatch. These names continue to work unchanged, but cannot be used together with the new name of the same sub-directive (e.g. url together with url_jobs).

Changed in version 3.38: Renamed the sub-directives from all, url, browser and command to all_jobs, url_jobs, browser_jobs and command_jobs.

Differ defaults

If you want to apply or change default settings for a differ, add them to the differ_defaults section in your config file. The following example will set the default model name to gemini-2.0-flash for the ai_google differ:

differ_defaults:
  _note: Default directives that are applied to individual differs.
  unified: {}
  ai_google:
    model: gemini-2.0-flash
  command: {}
  deepdiff: {}
  image: {}
  table: {}
  wdiff: {}

See differs for an explanation of the different differs and their directives.

Database configuration

The database section in your config file contains information on how snapshots are stored from run to run:

database:
  engine: sqlite3
  max_snapshots: 4

Database engine

engine

You can select one of the database engines as specified below; this can be overridden with the --cache-engine command line argument.

`sqlite3`

The default database engine, uses the sqlite3 database built into Python with data compression provided by msgpack. It is the most advanced solution due its speed due to indexing, small data files, and no data corruption or snapshot storage in case of an execution error.

The migration to this engine in version 3.2 allowed us to remove the requirement for the minidb Python package.

`textfiles`

Saves the latest snapshot of each job as its own individual text file. Only one snapshot can be saved, and both the ETag (allowing the speeding up of web data retrieval) and MIME type (enabling some diffing and reporting automation) will be lost.

`redis://...` or `rediss://...`

To use Redis as a database (cache) backend, specify a redis URI:

`mindib`

The deprecated legacy database engine, it is backwards compatible with urlwatch. Requires that the minidib Python package is installed; MIME types are not stored, is not indexed, data is not compressed, and the database file will grow indefinitely.

database:
  engine: redis://localhost:6379/

To use Redis, optional dependencies need to be installed; please see here

Note

Switching from Legacy (mindib) to Default (sqlite3) will cause an automatic data migration as long as the minidb Python package is installed; the old file database file is preserved for manual deletion. There is no migration path between any other databases types; for example, switching to Redis will create a new empty database at the first run.

`max_snapshots`

Maximum number of snapshots to save

Each time you run webchanges, it captures the data downloaded from the URL (or the output of the command specified), applies filters, and if it finds a change it saves the resulting snapshot to a database for future comparison. By default, only the last 4 changed snapshots are kept, but this number can be modified either in the configuration file or with the --max-snapshots command line argument.

If set to 0, all changed snapshots are retained (the database will grow indefinitely).

Note

Only applicable to the sqlite3 (default) database engine. When using redis or minidb database engines all snapshots will be kept (the database will grow indefinitely), while when using the textfiles database engine only the last snapshot is kept.

Tip

Changes (diffs) between saved snapshots can be redisplayed with the --test-differ command line argument (see here).

Added in version 3.11: For default sqlite3 database engine only.

Footnote

The top-level footnote directive is a string (or null) appended to the footer of every report (text, HTML and markdown). Use it to add a fixed disclaimer, a link, or contact info to all reports.

footnote: "Reports generated by webchanges -- contact admin@example.com"

If set to null (the default) or omitted, no footnote is added.

Version check

The top-level version_check directive controls the check made (at most once a day) against PyPi for a newer release of webchanges, whose result appears in report footers. Set its enabled sub-directive to false to disable the check entirely (no query is ever made):

version_check:
  enabled: false

The --check-new command line argument always queries PyPi, regardless of this setting.

Added in version 3.37.

Jobs and hooks files

The top-level jobs and hooks directives select the jobs and hooks files to use instead of the default jobs.yaml and hooks.py, so that you don’t need to type the --jobs and --hooks command line arguments at every run. Each takes a file, a glob pattern or a directory, or a list of these:

jobs: jobs  # every *.yaml file in the jobs directory and its subdirectories
hooks:
  - hooks.py
  - hooks/*.py

Relative paths are taken from the directory of the configuration file (not the current one), and ~ stands for the home directory.
A directory stands for every file in its tree with the .yaml (jobs) or .py (hooks) suffix; a glob pattern can include ** to match any number of subdirectories (e.g. jobs/**/*.yaml).
The files found are used in the order listed, with those matching the same pattern or directory in alphabetical order; with multiple jobs files, their contents are combined in that order.
The same smart file specification as the command line arguments applies (e.g. jobs: test will also look for test.yaml, jobs-test and jobs-test.yaml).
The --jobs and --hooks command line arguments override these directives.

If set to null (the default) or omitted, the default files are used.

Added in version 3.38.

Omitting configuration directives

When the config.yaml file is created, it contains all configuration directives and their default settings. If you omit/remove any directive from this file, webchanges will use the default value for the missing one. You can see a list of such omitted/missing directives and the default values assigned when running with the --vv or -vvv command line argument.

Remarks

YAML files do not allow for remarks; however, keys that start with underline are ignored and can be used for remarks.

Added in version 3.11.

Configuration

Overview

Display

Reports and Reporters

Job defaults

Handling of duplicate directives

Previous names

Differ defaults

Database configuration

Database engine

sqlite3

textfiles

redis://... or rediss://...

mindib

max_snapshots

Footnote

Version check

Jobs and hooks files

Omitting configuration directives

Remarks

`sqlite3`

`textfiles`

`redis://...` or `rediss://...`

`mindib`

`max_snapshots`