Field notes

What we learn running 464 scrapers in production — directory quirks, data cleaning, and the settings that decide whether a run finishes.

Guides4 Aug 20267 min

How to pull a full trade show exhibitor list

Exhibitor directories from Messe, RX, and Informa all publish the same data behind very different front ends. Here is how to get a complete list from any of them.

Read note

All notes

Research22 Jul 2026

Reading Amazon KDP niches with real BSR data

Best Seller Rank is the only demand signal Amazon gives away for free. Used carefully it tells you what a niche is worth; used carelessly it tells you nothing.

6 min

Workflow9 Jul 2026

From scraped rows to a CRM you can actually use

The gap between a scrape that finished and a lead list that converts is cleaning. Most of it comes down to four decisions about identity.

5 min

Engineering27 Jun 2026

What changes between job boards at scale

Naukri, Instahyre, and Zhaopin expose broadly the same fields, but salary, location, and freshness behave differently enough to break a shared schema.

8 min

Workflow11 Jun 2026

Getting scraped data into Sheets, CSV, and JSON

Each export format quietly changes your data. Knowing how is the difference between a clean handoff and a column of corrupted identifiers.

4 min

Engineering29 May 2026

Rate limits, retries, and staying unblocked

Most scrapers do not fail because a site defeated them. They fail because a run that worked once was scaled up without changing anything else.

6 min

Want one of these run on your own targets?

Book a call