ModelsDocsBlogEnterpriseLM LinkNew

Login or Signup

Models

Developer Docs

LM Link

Careers

Blog

Changelog

Enterprise Solutions

Privacy Policy

Terms of Use

👾 Element Labs, Inc. © 2026

LinkedIn GitHub Discord Twitter / X

Product

Download the app Models LM LinkNew LM Studio Hub Beta Releases Changelog

Developer

Developer Docs lmstudio-js lmstudio-python LM Studio CLI (lms)llms.txt llms-full.txt

Company

CareersWe're Hiring!

Enterprise Solutions

Legal

leglgnoml/crawler • LM Studio Hub

crawler

Public

Forked from bakit/crawler

crawler
/
src
/
embeddings

Project Files

src

cache

cache.ts

pageCache.ts

crawler

crawlPage.ts

extractContent.ts

fetchHtml.ts

normalizeUrl.ts

rankChunks.ts

embeddings

cosine.ts

embed.ts

search

providers

DuckDuckGoProvider.ts

SearchProvider.ts

duckduckgo.ts

services

crawlPage.ts

ranker.ts

retryFetch.ts

structuredExtraction.ts

tools

crawlSpecificPage.ts

crawlWebPage.ts

index.ts

searchWeb.ts

types

CrawlSession.ts

utils

blockedExtensions.ts

cleanText.ts

constants.ts

domainFilter.ts

summarizer.ts

index.ts

.gitignore

LICENSE

manifest.json

package-lock.json

package.json

README.md

runtime.js

tsconfig.json

src / embeddings /

...

cosine.ts

302 B

embed.ts

827 B