# Data normalization: one common schema for sources that look nothing alike.

Canonical URL: https://www.rastriq.es/en/data-normalization/ · Spanish version: https://www.rastriq.es/normalizacion-de-datos/

> We turn data from different sources into one common schema: taxonomy, entity matching, deduplication and QA that flags without filtering. With real rows.

Three car makers publish the same kind of price with three different structures. We turn it into a single table you can load, join and audit. Below, the real rows before and after.

QUICK ANSWER

**Quick answer:** Rastriq turns sources with different structures, such as the configurators of three car makers, into one common schema with documented types, source URL, capture date and quality flags that mark doubtful rows without filtering them out. Delivery is as CSV or JSON, to an S3 bucket or as an Apify dataset with API access.

3

Source schemas

1

Common schema

0

Rows dropped by QA

Get a normalized sample → See before and after

REAL SAMPLE

## Three schemas, one table

BMW publishes gross and net price with VAT in its own field. Kia mixes price, discount and a net-price flag. Volvo returns a base amount with a tax flag. All three describe the same concept using different names, types and semantics.

### Before · BMW (25 fields per row)

| market | model_name | line_name | fuel_type | currency | gross_list_price | net_list_price | vat_percentage |
|---|---|---|---|---|---|---|---|
| es | BMW 118d | Paquete M Sport Pro | D | EUR | 42768.1 | 34010.4 | 0.21 |
| es | BMW 118d | M Sport Design | D | EUR | 36673.7 | 29164.01 | 0.21 |

### Before · Kia (31 fields per row)

| country | model_name | trim_name | propulsion | currency | price | is_net_price | net_price | discount_amount |
|---|---|---|---|---|---|---|---|---|
| Spain | EV6 | Air | Electric | EUR | 35530.01 | false | 29363.64 | 13465 |
| Germany | EV6 | Air | Electric | EUR | 44990 | false | 44990 | 0 |

### Before · Volvo (13 fields per row)

| country | modelName | powertrain | basePriceAmount | basePriceCurrency | priceIncludesTax | priceStatus |
|---|---|---|---|---|---|---|
| ES | EX30 | BEV | 35800 | EUR | true | ok |
| SE | EX30 | BEV | 429000 | SEK | true | ok |

### After · common schema

| brand | market | model | trim | powertrain | currency | price_gross | price_net | discount_amount | qa_flags |
|---|---|---|---|---|---|---|---|---|---|
| BMW | ES | 118d | Paquete M Sport Pro | diesel | EUR | 42768.1 | 34010.4 | — | — |
| BMW | ES | 118d | M Sport Design | diesel | EUR | 36673.7 | 29164.01 | — | — |
| Kia | ES | EV6 | Air | bev | EUR | 35530.01 | 29363.64 | 13465 | — |
| Kia | DE | EV6 | Air | bev | EUR | 44990 | 44990 | 0 | net_equals_gross |
| Volvo | ES | EX30 | — | bev | EUR | 35800 | — | — | net_price_unavailable |
| Volvo | SE | EX30 | — | bev | SEK | 429000 | — | — | net_price_unavailable \| non_eur_currency |

Real rows from the BMW, Kia and Volvo samples. Amounts keep the source currency; values missing at the source stay empty and are flagged in qa_flags.

MAPPING

## How each field is translated

The mapping is documented field by field and versioned. If a manufacturer changes its structure, the change is recorded and the common schema does not break.

| Concept | BMW | Kia | Volvo | Common schema |
|---|---|---|---|---|
| Brand | constant per actor | constant per actor | constant per actor | brand |
| Market | market (es) | country (Spain) | country (ES) | market (ISO 3166) |
| Model | model_name | model_name | modelName | model |
| Trim | line_name | trim_name | edition | trim |
| Powertrain | fuel_type (D) | propulsion (Electric) | powertrain (BEV) | powertrain |
| Price incl. taxes | gross_list_price | price | basePriceAmount + priceIncludesTax | price_gross |
| Price excl. taxes | net_list_price | net_price | not published | price_net |
| Discount | not published | discount_amount | discountSavingAmount | discount_amount |

METHOD

## Four steps to unify sources

Normalization is not a one-off clean-up: it is a set of explicit rules that run on every delivery.

Taxonomy

We define closed lists for powertrain (diesel, petrol, bev, phev, hev), body type and drive, and map every source value. In the example, D becomes diesel and Electric or BEV become bev.

Entity matching

The same model shows up under different names: in the Volkswagen actor the same range is published as "Der Golf", "GOLF" and "Golf". We resolve those variants into a single entity with rules and review of doubtful cases.

Deduplication

We define a natural key (brand, market, model, trim, powertrain and capture date) and keep a single row per key. Repeats across runs do not inflate your series.

QA that flags and does not filter

Quality rules add a label in qa_flags but never remove rows. In the example, net_price_unavailable, non_eur_currency and net_equals_gross warn without hiding the data; you decide what to do with them.

COVERAGE

## Where we apply normalization

The same method works for any vertical with several sources. These are the cases with a public sample today.

OEM configurators

28 brands

Prices by market, trim and powertrain under a common schema across brands.

[See coverage →](https://www.rastriq.es/en/new-car-pricing-data/)

Spain used cars

Coches.net · Wallapop · Milanuncios

Year, mileage, fuel type, price and province aligned across three portals.

[See coverage →](https://www.rastriq.es/en/spain-used-car-data/)

Other sources

Under project

Machinery, tenders, reviews or directories: we define the common schema with you.

[See custom projects →](https://www.rastriq.es/en/managed-web-scraping/)

DELIVERY

## What you receive

The delivery includes data, documentation and rules. For the conceptual side, read the guide on [normalizing configurator pricing](https://www.rastriq.es/en/guides/normalize-car-configurator-pricing/).

Load-ready table

CSV or JSON in the common schema, plus a field dictionary with type, example and calculation rule.

Formats: CSV, JSON

Recurring delivery

Every scheduled run is already normalized and published to your S3 bucket or an Apify dataset with API access.

Channels: S3, Apify, API

Traceability

Each row keeps its source URL and capture date, and the mapping is versioned so you can reproduce any figure.

Fields: source_url, scraped_at

FAQ

## Frequently asked questions about normalization

### What is the difference between cleaning and normalizing?

Cleaning fixes isolated errors. Normalizing defines a schema, categories and stable rules so that different sources produce the same structure on every delivery and can be joined without manual work.

### Do you remove rows during QA?

No. QA tags rows with labels in qa_flags and lets them through. You decide on filtering, because a row that looks suspicious for one use case may be valid for another.

### Do you convert currencies?

By default we keep the source currency on every row. If you need a common currency, we add a converted column with a fixed, documented exchange rate, without replacing the original amount.

### Can you normalize my own files?

Yes, as long as they are product or market data that can be shared. We review a sample, define the target schema and apply the rules on every delivery.

### What happens if the source changes its structure?

We detect the change, update the mapping and version the rule. The common schema stays the same, so your loads do not break even if the manufacturer redesigns its configurator.

FREE SAMPLE

## Request a normalized sample

Tell us which sources you want to unify and we will return an extract in the common schema with the field dictionary.

- Normalized extract with qa_flags
- Field dictionary and per-source mapping
- No commitment

Received. Meanwhile, download the real samples at [/en/samples/](https://www.rastriq.es/en/samples/).

CONTACT

## Shall we talk about your schema?

Tell us which sources you handle and what table you need at the end.

[Get in touch →](https://www.rastriq.es/en/#contacto)

RELATED PAGES

## Keep exploring

[Buy datasets Catalogue with downloadable samples.](https://www.rastriq.es/en/buy-datasets/) [New car pricing data 28 brands under a common schema.](https://www.rastriq.es/en/new-car-pricing-data/) [Managed web scraping New extraction and recurring delivery.](https://www.rastriq.es/en/managed-web-scraping/)