Scan a whole site, paste a sitemap or add a single page, then choose exactly which pages your agent learns from
Most agents get most of their knowledge from a website. Open Knowledge, select Add knowledge, then Website.

Choose how much to read
Enter your homepage
Pick Entire site to read every page we can find, from your sitemap or by following links. If the URL has a path, This section only reads only the pages under it, so yoursite.com/docs reads your docs and nothing else.
Skip what you do not want
Add exclude chips for sections like /blog or /legal. They apply to this scan only; to skip pages for good, add an exclude rule.
Scan, then confirm
We scan first and read pages while the tab is open. A small site is learned with nothing to confirm. A large site shows what we found before anything is added, so you choose what to keep.
Paste the sitemap address
Enter the URL of your sitemap.xml. We read the list of pages from it instead of following links.
Review and confirm
From here it works like a whole-site scan: a small site is learned straight away, a large site asks first.
Enter the page address
Pick Just this page.
It is learned straight away
There is no scan step. Use it for one new article or a page a scan missed.
Scanning is free and never counts against your plan. Only the pages you keep do.
Large sites ask first
If your site is bigger than one scan reads, or would not fit your plan, we stop and show you:
- The real number. "We found 1,181 pages", not a capped count.
- What fits. We read a few pages to measure how much text this site puts on a page, then say roughly how many your plan holds.
- A starting selection. Pages visitors ask about (
/faqs,/about,/contact) come before the thousandth product listing. - Your call. Pages appear as a tree of folders; filter by path and tick or untick a folder at once.
Your website's pages
Each website has its own card under Sources on the Knowledge page. Choose Pages to see every page we know of, as a tree, with filters: All, Learned, Not added, Couldn't read and Excluded.
| Filter | What is in it | What you can do |
|---|---|---|
| Learned | The pages your agent uses, with how much text each holds | Choose Select, tick pages or a folder, then Remove. Open a page to see what your agent learned, Read it again, or edit its Suggested questions. |
| Not added | Pages we found that your agent has not read, marked Suggested, Visitors asked about this or New | Choose Select, tick pages, then Add, or Don't suggest to move them out of the way. Add N suggested adds every suggested page at once. |
| Couldn't read | Pages the last sync could not reach, with the reason (shown only when there are any) | Try all again, or remove a page that is gone from your site. |
| Excluded | Pages you removed and pages that match an exclude rule | Add and edit exclude rules. Include moves a page back to Not added. |
Exclude rules
On the Excluded filter, choose Add rule for pages your agent should never use.
| Rule | Example | Matches |
|---|---|---|
| Starts with | /blog | /blog and every page under it, like /blog/any-post |
| Contains | internal | Any page with that text in its path |
| Exact page | /old-pricing | That one page |
As you type, the rule shows how many pages it matches. Rules apply to every website of the agent and to every future scan, suggestion and sync. If a new rule matches pages your agent already learned from, you are asked first; saving it removes them.