Good price tracking is not a scraper that runs every night, but a process with a defined scope, matched products, comparable prices and alerts that someone reads. These are the steps.
Before extracting a single page, write down which decision the data will improve. It could be adjusting your tariff when a competitor cuts prices, spotting aggressive promotions in your category, validating the entry price of a new product or watching the used market to set residual values. Each decision implies a different frequency, level of detail and recipient. Tracking with no decision attached ends up as a dashboard nobody opens.
This guide follows the order in which we set up competitor price monitoring projects: scope, sources, matching, normalization, frequency, quality, alerts and delivery.
List the competitors, the products or categories and the markets that matter. It is better to start with a small, reliable scope than a huge, noisy one. A useful criterion: pick the products that concentrate your margin or volume and the three to five competitors your customer actually compares you with. Also note which price you care about: the shelf price, the offer price, the price with shipping or the base trim.
Not every price lives in the same place. Depending on the sector, the source changes:
The most underestimated problem in price tracking is deciding that two products are "the same". A car model has trims, engines and packs; an appliance has references that change by market; a used-car listing is unique. To match, define a stable key (brand, model, trim, year or manufacturer reference) and use exact matches on reliable fields before resorting to text similarity.
When you do use text, be careful. Substring detection produces false positives: searching for "ev" to find electric cars classifies "Clio evolution" as electric. Tokenize, compare whole words and, whenever it exists, rely on a structured field such as fuel. Record which signal decided each match so you can audit it. The guide to normalizing car configurator pricing develops this example.
A price without its basis is not comparable. Before computing anything, declare whether the amount includes VAT, whether it is list or discounted, which currency it is in and whether it includes costs such as shipping or registration taxes. In car configurators we already see the problem: Kia publishes msrp_before_discount, discount_amount and is_net_price, while BMW separates gross_list_price, net_list_price and total_taxes. Until they are translated to a common schema, comparing brands means comparing tax bases.
Always keep the original value next to the normalized one. That way, if a conversion rule turns out to be wrong, you can recompute without extracting again.
The right frequency depends on how fast the price changes and what it costs to react late. A manufacturer tariff changes rarely, so weekly or monthly runs are enough; marketplace prices with flash offers may need daily or more frequent runs; used-car listings are watched to detect new, removed and repriced items. Running more often than needed only adds noise and cost, and running less often means you arrive late to the movement you wanted to see.
With a stable frequency you accumulate history, and history is what turns a single price into a series: you can see trends, seasonality and the effect of promotions. If you are not storing it yet, start today.
Web data carries noise: zero prices, mixed currencies, duplicates, expired listings and format changes. Two possible policies are to filter or to flag. Our recommendation is to flag and let through: add a quality flag to each doubtful record and let whoever consumes the data decide. If you filter silently, a competitor that disappears from the data looks like it raised its price or stopped selling, and nobody will know.
Add a volume alert: if a run returns noticeably fewer records than the previous one, the source has probably changed, not the market. That check catches broken extractors sooner than any other.
Value arrives when the data reaches whoever decides. Define the events that deserve an alert (a competitor drops below a threshold, a new product appears, one disappears), and deliver the information in the format your team already uses: a scheduled CSV or JSON, an API, a bucket or an integration with your analytics tool. Avoid alerts for every minimal change; an alert sent every day stops being read.
If you would rather not operate the chain, the managed web scraping service covers access, maintenance and delivery, and on buy datasets you can see which datasets are already extracted.
Work with public product data and exclude personal data: phone numbers, emails, names of private sellers and the like are not needed to watch prices and create risk. Respect each source terms of use and check your provider data licence to know which uses are covered. None of this replaces legal advice for your specific case, but a process that starts from product data, with the source noted on every record, is a far sounder basis.
The quickest way to know whether a dataset is useful is to look at real rows. On samples you can download files of up to 100 rows, in JSON and CSV, for each source, already sanitized of personal data. Compare them with your products and competitors, and if the data answers your question, move to recurring tracking.
It depends on how fast they change. For list prices a weekly or monthly run is usually enough; for marketplaces with offers, daily or more often. What matters is that the frequency is stable so you can accumulate history and compare series.
Define a stable key, such as brand, model, trim and year, and match on exact fields before using text similarity. Record which signal decided each match and avoid substrings, which produce false positives such as "ev" inside "Clio evolution".
Better to flag them and let them through. A quality flag lets the analyst decide, whereas a silent filter makes a missing data point look like a market change. A zero price or an unexpected currency is annotated, not deleted.
Phone numbers, emails, names of private sellers, user identifiers and the like. They are not needed to monitor prices and they create risk. Our samples exclude them and the data licence details what is delivered.
Yes. Rastriq can operate extraction, matching, normalization and delivery, or supply datasets already extracted. You can start with a sample and define sources, frequency and format afterwards.
Download a real sample of up to 100 rows, already sanitized, or tell us which sources you need.