Oaida Adrian

Most data jobs re-pull the whole source every run and waste time and money on data that didn't move. A change-detection engine wakes up, checks what actually changed, and pushes only the diff. I built a generic one: point it at any URL — a JSON feed, an RSS/Atom feed, or a plain HTML page — and it emits only the changes, with before and after values, to the dataset and/or your webhook.

Raw web pages are terrible training data. Nav bars, cookie banners, "related articles" and ads drown the signal, and near-identical syndicated text pollutes the corpus. If you're fine-tuning a domain model — legal reasoning, medical QA, financial analysis — you want clean vertical text with provenance, not a pile of HTML. I built an actor that goes from seed URLs to a token-aware JSONL dataset in one run.

For a long time I built B2B lead-generation tools the way everyone does: scrape a directory, fight the anti-bot, re-scrape when the layout changes, pray the data is current. Then I found out the Romanian government publishes the entire company registry as open data — 4.2 million firms, refreshed monthly, with legal status, activity codes and directors. No scraping required.