Data normalization: one common schema for sources that look nothing alike.

Three car makers publish the same kind of price with three different structures. We turn it into a single table you can load, join and audit. Below, the real rows before and after.

QUICK ANSWER

Quick answer: Rastriq turns sources with different structures, such as the configurators of three car makers, into one common schema with documented types, source URL, capture date and quality flags that mark doubtful rows without filtering them out. Delivery is as CSV or JSON, to an S3 bucket or as an Apify dataset with API access.

Source schemas

Common schema

Rows dropped by QA

Get a normalized sample →See before and after

REAL SAMPLE

Three schemas, one table

BMW publishes gross and net price with VAT in its own field. Kia mixes price, discount and a net-price flag. Volvo returns a base amount with a tax flag. All three describe the same concept using different names, types and semantics.

Before · BMW (25 fields per row)

market	model_name	line_name	fuel_type	currency	gross_list_price	net_list_price	vat_percentage
es	BMW 118d	Paquete M Sport Pro	D	EUR	42768.1	34010.4	0.21
es	BMW 118d	M Sport Design	D	EUR	36673.7	29164.01	0.21

Before · Kia (31 fields per row)

country	model_name	trim_name	propulsion	currency	price	is_net_price	net_price	discount_amount
Spain	EV6	Air	Electric	EUR	35530.01	false	29363.64	13465
Germany	EV6	Air	Electric	EUR	44990	false	44990	0

Before · Volvo (13 fields per row)

country	modelName	powertrain	basePriceAmount	basePriceCurrency	priceIncludesTax	priceStatus
ES	EX30	BEV	35800	EUR	true	ok
SE	EX30	BEV	429000	SEK	true	ok

After · common schema

brand	market	model	trim	powertrain	currency	price_gross	price_net	discount_amount	qa_flags
BMW	ES	118d	Paquete M Sport Pro	diesel	EUR	42768.1	34010.4	—	—
BMW	ES	118d	M Sport Design	diesel	EUR	36673.7	29164.01	—	—
Kia	ES	EV6	Air	bev	EUR	35530.01	29363.64	13465	—
Kia	DE	EV6	Air	bev	EUR	44990	44990	0	net_equals_gross
Volvo	ES	EX30	—	bev	EUR	35800	—	—	net_price_unavailable
Volvo	SE	EX30	—	bev	SEK	429000	—	—	net_price_unavailable \| non_eur_currency

Real rows from the BMW, Kia and Volvo samples. Amounts keep the source currency; values missing at the source stay empty and are flagged in qa_flags.

MAPPING

How each field is translated

The mapping is documented field by field and versioned. If a manufacturer changes its structure, the change is recorded and the common schema does not break.

Concept	BMW	Kia	Volvo	Common schema
Brand	constant per actor	constant per actor	constant per actor	brand
Market	market (es)	country (Spain)	country (ES)	market (ISO 3166)
Model	model_name	model_name	modelName	model
Trim	line_name	trim_name	edition	trim
Powertrain	fuel_type (D)	propulsion (Electric)	powertrain (BEV)	powertrain
Price incl. taxes	gross_list_price	price	basePriceAmount + priceIncludesTax	price_gross
Price excl. taxes	net_list_price	net_price	not published	price_net
Discount	not published	discount_amount	discountSavingAmount	discount_amount

METHOD

Four steps to unify sources

Normalization is not a one-off clean-up: it is a set of explicit rules that run on every delivery.

Taxonomy

We define closed lists for powertrain (diesel, petrol, bev, phev, hev), body type and drive, and map every source value. In the example, D becomes diesel and Electric or BEV become bev.

Entity matching

The same model shows up under different names: in the Volkswagen actor the same range is published as "Der Golf", "GOLF" and "Golf". We resolve those variants into a single entity with rules and review of doubtful cases.

Deduplication

We define a natural key (brand, market, model, trim, powertrain and capture date) and keep a single row per key. Repeats across runs do not inflate your series.

QA that flags and does not filter

Quality rules add a label in qa_flags but never remove rows. In the example, net_price_unavailable, non_eur_currency and net_equals_gross warn without hiding the data; you decide what to do with them.

COVERAGE

Where we apply normalization

The same method works for any vertical with several sources. These are the cases with a public sample today.

OEM configurators

28 brands

Prices by market, trim and powertrain under a common schema across brands.

See coverage →

Spain used cars

Coches.net · Wallapop · Milanuncios

Year, mileage, fuel type, price and province aligned across three portals.

See coverage →

Other sources

Under project

Machinery, tenders, reviews or directories: we define the common schema with you.

See custom projects →

DELIVERY

What you receive

The delivery includes data, documentation and rules. For the conceptual side, read the guide on normalizing configurator pricing.

Load-ready table

CSV or JSON in the common schema, plus a field dictionary with type, example and calculation rule.

Formats: CSV, JSON

Recurring delivery

Every scheduled run is already normalized and published to your S3 bucket or an Apify dataset with API access.

Channels: S3, Apify, API

Traceability

Each row keeps its source URL and capture date, and the mapping is versioned so you can reproduce any figure.

Fields: source_url, scraped_at

FAQ

Frequently asked questions about normalization

What is the difference between cleaning and normalizing?

Cleaning fixes isolated errors. Normalizing defines a schema, categories and stable rules so that different sources produce the same structure on every delivery and can be joined without manual work.

Do you remove rows during QA?

No. QA tags rows with labels in qa_flags and lets them through. You decide on filtering, because a row that looks suspicious for one use case may be valid for another.

Do you convert currencies?

By default we keep the source currency on every row. If you need a common currency, we add a converted column with a fixed, documented exchange rate, without replacing the original amount.

Can you normalize my own files?

Yes, as long as they are product or market data that can be shared. We review a sample, define the target schema and apply the rules on every delivery.

What happens if the source changes its structure?

We detect the change, update the mapping and version the rule. The common schema stays the same, so your loads do not break even if the manufacturer redesigns its configurator.

FREE SAMPLE

Request a normalized sample

Tell us which sources you want to unify and we will return an extract in the common schema with the field dictionary.

Normalized extract with qa_flags
Field dictionary and per-source mapping
No commitment

Received. Meanwhile, download the real samples at /en/samples/.

CONTACT

Shall we talk about your schema?

Tell us which sources you handle and what table you need at the end.

Get in touch →