What the crawler does
Kuykendall Industries, LLC operates KISearchBot to discover public web pages for KI Search. People and software using the hosted search engine can find indexed pages and follow their links back to the publisher. Discovery does not guarantee inclusion, ranking, or a particular recrawl time.
The crawler requests public HTTPS pages without signing in. It does not use account cookies or run page scripts. Search indexing, offline redistribution, and model training are separate uses; being publicly accessible does not grant permission for all three.
Recognize a request
KISearchBot/1.0 (+https://search.kuykendall.industries/bot; contact@forwarding.kuykendall.industries)
The crawler uses the product token KISearchBot when reading robots.txt. A user-agent string is an identifier, not proof of the sender’s identity. The scheduled worker signs requests with Ed25519 HTTP Message Signatures. Its public key directory supports verification without trusting the user-agent string. Signature availability does not imply acceptance in a publisher’s verified-bot program. No fixed crawler IP range is claimed.
Search, AI input, and reuse
Permission to appear in search does not grant permission for AI overviews, offline redistribution, or model training. Publisher instructions and separately documented source grants govern each use. Pages marked noarchive may appear as links with a short permitted excerpt; they have no saved-passage view and are excluded from AI overviews. KI Search does not turn a crawler response or an ai-train=yes signal into a training license.
News results distinguish reporting, analysis, opinion, and government statements. Publication and update dates come from the source; the verification date records a crawler check. Structured financial citations preserve the filing accession or dataset, observation period, and available vintage.
Control crawling
To exclude your entire site, publish this group in /robots.txt on the relevant host. To exclude a section, replace the slash with its path.
User-agent: KISearchBot Disallow: /
KISearchBot checks each destination’s crawl rules before requesting pages. A robots exclusion controls crawler access; it is not an access-control mechanism for confidential content.
Make pages discoverable
Link to a page from another public page, list it in a sitemap on the same HTTPS origin, or advertise a supported RSS or Atom feed in the page’s HTML head. The crawler can follow those feed links to retrieve articles. Advertise a sitemap in your robots.txt:
Sitemap: https://example.org/sitemap.xml
Discovery and recrawling are bounded. A sitemap or feed helps discovery but does not promise immediate indexing.
Control indexing and excerpts
Use a robots meta tag or an X-Robots-Tag HTTP header with noindex to request exclusion when the crawler next reads the page. Use nofollow to stop link discovery from that page. Use nosnippet to withhold excerpts, or max-snippet to limit their length.
<meta name="robots" content="noindex">
These directives can be read only when crawling is allowed. Changes are processed on a later crawl; for a correction or removal request that needs attention before then, contact KI below.
Corrections and removal
Use the KI privacy and data-rights form to report a source, request correction or removal, or raise a crawler issue. Include the affected public URL and your relationship to the site. Do not send passwords or private documents.
Search privacy explains information handling and your rights. Search terms explains use of the service and source material.

