Diffbot

Diffbot builds AI models that read and transform the unstructured web into knowledge. It links facts in a graph-like structure, enabling intelligent AI applications such as the Knowledge Graph and GraphRAG. Diffbot is known for the Extract API, the automatic web scraper that doesn’t use any rules, and is used to access unstructured data from the public web at scale.

Diffbot APIs make it possible to classify and extract meaningful entities from existing web pages — the Extract API. They can find meaningful entities from within entire websites — the Crawl API. They enable searching the public web as a humongous graph database — the Search (DQL) API. They allow enhancing existing data with public web data — the Enhance API. They identify entities and classify the context of raw text — the Natural Language API.

Diffbot APIs can be combined to power intelligent automated systems, such as an automated sanctions tracker. A good starting point is adding diffbot-skills to an agent harness and prompting it for a tour. The site contains the entire API Reference, including Authentication to set up your token, then pick one of the APIs to start structuring the web.

Diffbot operates its crawl of the entire public web and maintains the world's largest automated Knowledge Graph. It emphasizes a non-rule-based approach to extraction and provides tools for developers to build autonomous systems using web data.

Use cases

• Use Diffbot to tap into unstructured data from the public web at immense scale.