4 stations · 14 tools

AI tools for desk research and competitor analysis

Tools for discovering, collecting and analysing publicly available information. Explore markets, compare competitors and gather evidence for your research.

Web Scraping

Automatically collect data from websites: prices, product range, competitor listings.

Browse AI

Set up a no-code “robot” that collects data from websites (prices, product pages, lists) and tracks changes.

Website
Web ScrapingYesNo
Diffbot

An API that turns web pages into structured data: articles, products, discussions. Crawls whole sites, extracts entities and sentiment, and offers a knowledge graph of companies and people.

Website
Web ScrapingYesOfficial
Firecrawl

Developer service: scrapes pages and entire sites, searches the web, parses PDF and DOCX; returns clean text or structured data by your field schema.

Website
Web ScrapingYesOfficial
Thunderbit

Chrome extension: open a site, click a button — page data lands in a table, no setup or code. Exports to Excel, Google Sheets, Airtable, Notion; scheduled scraping available.

Website
Web ScrapingYesOfficial
Kadoa

Describe in plain words what data you need from public websites (filings, prices, job posts); it scrapes on a schedule, sends alerts and exports to Excel, Google Sheets, Slack or API.

Website
Web ScrapingNoNo
Browse AI

Set up a no-code “robot” that collects data from websites (prices, product pages, lists) and tracks changes.

Free plan

Web Scraping

Free · paid from $48/mo

Website
Diffbot

An API that turns web pages into structured data: articles, products, discussions. Crawls whole sites, extracts entities and sentiment, and offers a knowledge graph of companies and people.

Free planMCP server

Web Scraping

Free · paid from $299/mo

Website
Firecrawl

Developer service: scrapes pages and entire sites, searches the web, parses PDF and DOCX; returns clean text or structured data by your field schema.

Free planMCP server

Web Scraping

Free · paid from $19/mo

Website
Thunderbit

Chrome extension: open a site, click a button — page data lands in a table, no setup or code. Exports to Excel, Google Sheets, Airtable, Notion; scheduled scraping available.

Free planMCP server

Web Scraping

Free · paid from $15/mo

Website
Kadoa

Describe in plain words what data you need from public websites (filings, prices, job posts); it scrapes on a schedule, sends alerts and exports to Excel, Google Sheets, Slack or API.

Web Scraping

on request

Website

Questions and answers

Which AI search engine cites its sources for market research?

AI search engines that search the web in real time and link every answer to its sources are the safest choice for market research, because each claim can be checked. Perplexity links every claim to a source, offers a Deep Research mode for long-form topic overviews, has a free tier and an official MCP server. ChatGPT, Gemini and Claude can also search the web and have free tiers, and work well for summarizing uploaded reports, but always check AI output.

How can I estimate a competitor's website traffic and top keywords?

Use a traffic estimation tool: it shows approximate visits and ranking keywords for any domain, without access to the competitor's own analytics. Similarweb shows total visitors, countries and channels (search, ads, social, direct). Semrush combines traffic by channel with SEO and ad data and has a limited free account. Ahrefs focuses on Google keywords and backlinks and shows search traffic only. Google Trends is free for comparing search interest over time.

How do I monitor brand mentions across social media, news and AI chatbots?

Social listening tools collect mentions of your brand and competitors and use AI to score sentiment (always check AI output), while SEO suites now also track how often AI chatbots mention a brand. Brand24 covers social media, news, blogs and forums, measures reach and sentiment, and has an official MCP server. For mentions in chatbot answers, Ahrefs offers Brand Radar, and both Semrush and Similarweb track brand mentions in AI responses.

Can I scrape competitor prices and product data without coding?

Yes, several no-code scrapers turn web pages into tables without writing code. Thunderbit is a Chrome extension: click a button and page data goes to Excel, Google Sheets, Airtable or Notion; it has a free tier with a monthly page limit. Browse AI lets you train a robot that collects prices and tracks changes, with a free tier. Kadoa takes a plain-language description and scrapes on a schedule. Firecrawl and Diffbot are APIs that need a developer.

Other research lines