DispatchListening is how Dispatch reads the public feeds that newsrooms choose to follow, such as a police department's news releases, a city's RSS feed, a YouTube channel or a Bluesky account. When a newsroom adds one of your feeds, DispatchListening checks it for new posts so the newsroom sees them in one place.
It is a reader for feeds that people subscribed to, not a crawler. It reads the address a newsroom gave it, a few likely feed addresses when that address is a web page, and your robots.txt: it never follows the links in a feed, never walks the pages of your site and never builds a search index.
Every request it makes to your site carries this user agent:
DispatchListening/1.0 (+https://dispatchsocial.app/bot)It never presents itself as a web browser, and it never changes its name, even after a site turns it away.
rel="alternate" link in its head. If it points to none, DispatchListening tries a few common feed addresses on the same site, such as /feed/ and /rss.xml, and stops after a handful of requests.Cache-Control: max-age, these regular checks wait at least that long, up to 30 minutes.Retry-After or a refusal from your site, or after it has stopped reading a feed, so the waits below hold for checks by hand too.If-None-Match and If-Modified-Since whenever your server gave an ETag or a Last-Modified date, so an unchanged feed can answer 304 Not Modified without sending the feed again.Retry-After, DispatchListening waits that long before it asks again, up to the same 6 hours.DispatchListening follows robots.txt. To keep it off your whole site, add this group:
User-agent: DispatchListening
Disallow: /To keep it away from one feed, disallow that feed's path instead. It follows the group that names DispatchListening, or the * group when no group does. The longest matching rule wins, Allow wins a tie, and the * and $ wildcards work. A site without a robots.txt is read as usual.
It reads robots.txt when a newsroom adds a feed and again about once a day, so a new rule usually takes effect within a day. The newsroom is told that your site asks automated readers not to read that feed, and Dispatch stops reading the feed. It keeps reading your robots.txt about once a day, in case the rule changes.
Redirects are checked too: if a feed's address starts redirecting somewhere your robots.txt disallows, Dispatch stops reading that feed.
On Bluesky, an account that asks apps not to show it to people who aren't logged in isn't followed, and Dispatch stops checking an account within about a day after it turns that setting on. For 3 days after DispatchListening first sees a Bluesky post, Dispatch rechecks it from time to time and removes it if it was deleted. Anything it already keeps otherwise goes on the schedule below, or sooner if you ask.
Only the newsrooms that follow a source see the posts DispatchListening reads from it: in Dispatch, and in the alerts each newsroom chooses to get by email, in Slack or at its own web address. Dispatch deletes the posts it keeps about 30 days after DispatchListening first saw them. Alerts already delivered stay wherever the newsroom received them.
To ask Dispatch to stop checking a feed or an account, or to delete the posts it keeps from one, email [email protected] with the address of the feed or account. Questions about DispatchListening go to the same address.