Upgrading from urlwatch

webchanges reads your urlwatch jobs and configuration files as they are, so you keep every job you have built and gain clearer reports, AI summaries, image and structured-data diffs, and an HTTP client that gets past bot walls. It began as a fork of urlwatch 2.21 and has been developed independently since 2020, while staying compatible with urlwatch’s files: most setups need no edits at all.

For a feature-by-feature table, see the webchanges vs urlwatch comparison.

Why switch

Reports you can read at a glance

  • HTML reports keep the page’s formatting (bold, italics, underlining, headers, bulleted lists and indentation) and its links stay clickable.

  • Added and deleted lines are marked with color and strikethrough, readable in dark mode and in email clients that strip stylesheets, such as Gmail.

  • Long lines wrap instead of being cut off.

  • The table differ shows changes side by side, and the wdiff differ highlights the words that changed within a line.

  • The telegram reporter keeps links, bold, italics, underlining and strikethrough.

https://raw.githubusercontent.com/mborsetti/webchanges/main/docs/html_diff_filters_example_1.png

https://raw.githubusercontent.com/mborsetti/webchanges/main/docs/html_diff_filters_example_3.png

Changes worth being told about

  • The ai_google differ summarizes a long diff (a contract, a policy, terms of service) in a paragraph of plain language, using Google’s Gemini.

  • The image differ watches images and reports an image with the differences highlighted.

  • The deepdiff differ compares JSON, YAML and XML element by element, so that reordered list items, reformatting or noise in the last decimal don’t raise false alarms.

  • additions_only reports only what was added, such as new items in a list, and deletions_only only what was removed.

Sites that block urlwatch

  • Choose the HTTP client per job with http_client: httpx (with HTTP/2), requests, or curl_cffi, which presents the TLS fingerprint of a real browser to get past Cloudflare, Akamai and similar bot walls.

  • Browser jobs run Chrome, Edge, Firefox or WebKit through Playwright, can wait for a URL, a selector or a JavaScript condition, block images and other resources, and can attach to a browser that is already running (connect_over_cdp).

  • same_site_delay spaces out jobs that hit the same site, to stay clear of rate limits.

Fewer false alarms

  • empty_as_error treats an empty response as a transient error instead of reporting the page’s content disappearing and then reappearing.

  • suppress_repeated_errors reports an error once, not on every run, and suppress_error_ended skips the report that an error has ended when the content is unchanged.

  • ignore_connection_errors, ignore_timeout_errors, ignore_http_error_codes and the other ignore_* directives also work for browser jobs.

Snapshots you can trust

  • Snapshots are saved only after the reports have been sent, so a crash or a reporter error no longer loses a change.

  • The default SQLite database is indexed and compressed and keeps the latest 4 snapshots of each job (set with –max-snapshots), so it no longer grows without limit.

  • Undo mistakes with –rollback-database and –delete-snapshot.

Tools for maintaining your jobs

  • –errors runs every job and lists those that fail or whose filters return nothing.

  • –test-differ replays the diffs of every snapshot saved, and –test without a job checks your jobs and configuration files for errors.

  • Editors such as VS Code autocomplete and validate jobs.yaml and config.yaml from the JSON Schemas that webchanges installs next to them.

  • The !env tag reads secrets from environment variables instead of storing them in your files.

  • Send different jobs to different destinations by configuring a reporter several times, each with its own tags, and open a GitHub issue with the GitHub Issue reporter.

Actively maintained

Releases ship regularly, supporting Python 3.12 to 3.15 (including the free-threaded build), and every release is tested on Linux, macOS and Windows.

Switching from urlwatch

  1. Confirm that the latest release of urlwatch (2.29) runs your jobs without errors. webchanges reads the same files, so fix any error there first.

  2. Install webchanges. If any of your jobs uses a browser (it has a navigate directive), install the use_browser extra:

    uv pip install webchanges               # or: uv pip install webchanges[use_browser]

    Browser jobs use Google Chrome; if it is not installed, run webchanges --install-chrome.

  3. Check your files:

    webchanges --test                        # check the jobs and configuration files for errors

    This first run copies your urlwatch files into webchanges’ own directory: urls.yaml becomes jobs.yaml, urlwatch.yaml becomes config.yaml, hooks.py keeps its name, and the snapshot database cache.db becomes snapshots.db. webchanges looks for them where urlwatch keeps them (~/.urlwatch, or else its default directories), and copies only what it doesn’t have yet. The originals are left untouched, so urlwatch keeps working and you can switch back at any time.

  4. Fix what --test reports, using What you may need to change. Renamed directives keep working and only print a deprecation warning, which you can address when convenient.

  5. Run webchanges. On the first run it converts its copy of urlwatch’s snapshot database to its own format, which needs the minidb package for this one conversion:

    uv pip install minidb                     # needed once, for the conversion
webchanges                                # run all jobs

    You can uninstall minidb afterwards. Don’t point --database at urlwatch’s legacy cache.db instead of letting webchanges upgrade it into its own separate copy.

  6. All of the existing files are kept; if you don’t like what you see, you can keep on using urlwatch.

If you encounter a problem or have a suggestion, open an issue on GitHub.

What you may need to change

This section lists every difference, relative to urlwatch 2.29, that can require you to edit your files or change how you run the program.

Changes that stop a job or the run

Change these before your first run. (webchanges also rejects any directive it does not recognize, instead of ignoring it, so that a typo cannot go unnoticed.)

In urlwatch

In webchanges

urlwatch --tags work home

webchanges --tags work,home: separate tags with commas (see Select subset of jobs by tag).

hooks.py custom code

Replace every urlwatch import with the same import from webchanges, then adapt your classes to the hooks API: for example, a filter’s filter() method receives the data’s media type and returns a (data, mime_type) tuple, and a job’s retrieve() method returns (data, etag, mime_type).

Renamed names that still work

These names were changed for clarity, or replaced by a more capable equivalent. The old ones keep working (most print a deprecation warning), so you can update them when convenient:

urlwatch name

webchanges name

navigate: <URL>

url: <URL> and use_browser: true

useragent: <agent> (navigate jobs)

A User-Agent: <agent> header in headers

browser: firefox, browser: webkit (navigate jobs)

use_browser: firefox, use_browser: webkit (see use_browser)

wait_for: <selector> (navigate jobs)

wait_for_selector: <selector>

filter, diff_filter

filters, diff_filters

diff_tool: <command>

differ: with name: command and command: <command> (see Differs)

http_proxy, https_proxy

proxy

json: true

data_as_json (see data_as_json)

kind: shell

kind: command

stderr: urlwatch (command jobs)

stderr: forward

grep, grepi filters

keep_lines_containing, delete_lines_containing

html2text methods pyhtml2text, re

html2text (the default), strip_tags

html2text method lynx

The default method, ignoring lynx’s options. Its text is formatted differently, so the first run reports a change; to keep lynx’s output, run it with the execute filter instead

job_defaults keys all, url, browser, shell

all_jobs, url_jobs, browser_jobs, command_jobs

--urls, --cache, --gc-cache

--jobs, --database, --gc-database

--test-filter, --test-diff-filter

--test, --test-differ

--list

--list-jobs (--list is accepted as an abbreviation)

slack reporter

Webhook (Slack, Mattermost etc.) (same sub-directives)

mattermost reporter

Webhook (Slack, Mattermost etc.) with markdown: true

shell reporter

run_command, which inserts the report into the command line with {text} instead of sending it on standard input

gotify sub-directive title

subject

Behavior that differs

None of these need an edit, but they change what you see:

  • The database format has been upgraded for speed, size, and to hold more detail. To keep using urlwatch’s database format, run with –database-engine minidb (requires the minidb package).

  • The html2text filter without a method converts HTML to Markdown, while urlwatch’s default (re) only removed the tags; and its html2text method uses the Python html2text package, not the html2text program. Jobs using either report a change on their first run. To get the closest to urlwatch’s default output, add method: strip_tags.

  • The database keeps the latest 4 snapshots of each job. To keep them all, as urlwatch does, set max_snapshots to 0 in config.yaml or run with --max-snapshots 0.

  • Jobs in reports are sorted alphabetically by name.

  • The telegram reporter’s monospace sub-directive has no effect, as reports are sent in Markdown; it is reported as unrecognized.

  • When run from a terminal, --enable, --disable and --delete ask for confirmation before rewriting the jobs file, which (as in urlwatch) loses all its remarks (see Enable or disable a job).

  • A job selected on the command line by its URL, command or name must match one job only; if it matches several, the error lists them and you select one by its index number instead (see Select subset of jobs).

  • On Windows with Python 3.12 to 3.14, a UnicodeEncodeError from a command job or the shellpipe filter is fixed by running Python in UTF-8 mode (the default from Python 3.15).