Website Crawler Sitemap

Overview

The Website Crawler Sitemap source connector allows you to extract site structure and visual relationships between pages from Websites into your desired vector database.

Configuration Options

• Name: This field represents the name you want to assign to the actor instance responsible for managing the Website Crawler Sitemap source. Choose a descriptive and unique name to easily identify this instance within your data activation tool (dat).

• Site URL: Enter the Site URL. like https://www.example.com.

• Sitemap URL: If available, enter the path to the sitemap. If left untouched, sitemap will be attempted to read from https://www.example.com/sitemap.

Supported streams

The following streams are supported for this source:

• crawler_sitemap