Competitor pricing, tracked properly
Price monitoring is the most commonly attempted scraping project and the one most often abandoned. Collecting a price is trivial. Knowing that a competitor's listing is the same product as yours, that the price you captured was the one a real customer would see, and that a change is worth an alert rather than noise, is the actual work. Most tools fail on the matching and everyone notices about a month in.
- Starting at
- from £1,200
- Refresh
- Daily, or hourly for volatile categories
- Access
- public web
What comes out of it
- Listed price, promotional price, and unit price
- Stock status and estimated availability
- Product identifiers, title, brand, and variant
- Promotion text and any member or bulk pricing
- Capture timestamp and the region served
What makes Retailer and marketplace pricing awkward
Displayed price is often not the real price. Promotions, member pricing, regional variation, and cart-level discounts mean the figure in the markup can differ from what a customer pays, and capturing the wrong one produces confidently incorrect intelligence. Handling that means capturing price alongside its context: promotion flags, unit price, availability, and the region the page was served for.
Product matching is the whole game and nobody sells it that way. Two retailers describe the same item with different titles, different pack sizes, and often no shared identifier, so matching has to combine identifiers where present with title, brand, and attribute similarity where absent. A price feed built on bad matching is worse than no feed, because it produces confident numbers about the wrong products.
Matching before monitoring
Every engagement starts with matching your catalogue to competitor listings, because monitoring an unmatched catalogue produces numbers nobody trusts. Where shared identifiers exist the work is straightforward. Where they do not, matching combines brand, normalised title, pack size, and attributes into a similarity score.
Matches above a high threshold are accepted, matches in the middle go to a review queue for a human decision, and low matches are rejected. That review step is usually a few hours of someone's time at the start and it is what makes the resulting data trustworthy for the following year.
Once matched, pairs are stored and reused, so the expensive part happens once rather than on every run.
Capturing a price that means something
Each capture records not just a number but its context: whether a promotion was running, what the unit price was, whether the item was in stock, and which region the page was served for. Without that context a price history is misleading, because a drop might be a genuine repositioning or a three-day promotion that ended.
Change detection then fires on meaningful movements rather than every fluctuation. A threshold on percentage change, a rule about promotional versus base price, and a minimum duration before alerting turn a noisy stream into a handful of notifications someone will actually read.
Stock tracking is often more valuable than price. A competitor out of stock on a product you hold is a short-lived commercial opportunity, and it is the kind of thing worth an immediate alert rather than a daily digest.
What it costs
Workflow automation starts from £1,200 for a bounded price extraction from one permitted retailer and an agreed product sample. Multi-site matching, scheduled capture, historical reporting and alerts are priced after the scope is defined.
A full system with catalogue-wide matching, a review workflow, promotion-aware history, and dashboard reporting is quoted separately against the agreed sources, delivery requirements and acceptance checks. Most of the variable cost is matching volume and how well-identified your catalogue is.
Retail sites change layout frequently and some deploy bot protection that escalates over time, so this is a category where maintenance is not optional. Plans start at £400 a month. I will also tell you before starting if a target site's protections make reliable collection unlikely, rather than taking the work and delivering something that breaks in week three.
Where the line is
Publicly accessible pages, robots.txt and terms respected.
I collect publicly displayed prices from pages any customer can see, respect robots.txt, and stay within rate limits that do not degrade the target site. I do not build systems that create accounts, defeat bot protection, or access anything behind a login. Where a retailer's terms forbid automated collection I will say so before quoting rather than after. Pricing facts themselves are generally not protected, but the manner of collection is where the exposure sits, so that is where the care goes.
MathersLabs is not affiliated with or endorsed by Retailer and marketplace pricing. Work is scoped to publicly accessible data and to compliance with applicable terms and law, and feasibility is assessed per engagement before any fixed price is agreed.
Related work
AussieBankStatements
Bank statement converter and categorizer that never stores what it reads
A custom full-stack app on GCP that parses Australian bank statement PDFs into clean CSV. No AI, a dedicated parser per bank, and nothing written to disk. Built for people who want the extraction without handing their spending history to a chatbot that keeps it.
SignalSauce
250 qualified leads a month, fully automated
Built an end-to-end intelligence pipeline that ingests from 5 curated news sources twice daily, scores every announcement against a custom algorithm, and enriches the ones that pass with 21 structured data points per company.
Questions people ask first
Is your source on this list?
Tell me what data you need and where it lives. I will tell you whether it is feasible, what it would take, and whether it is worth building before you commit to anything. Working hours overlap with US Eastern and Pacific.