Data Sources & Methodology

Overview

TenderWatch aggregates public-sector procurement data from official national and EU-level sources. All data ingested into TenderWatch originates from publicly accessible feeds published by government bodies under their transparency obligations. We do not scrape private systems, internal procurement portals, or anything behind authentication.

We believe transparency in methodology outweighs any external certification badge for our use case. This page documents exactly where data comes from, how often it's refreshed, how it's validated, and how to report errors.

Active data sources

1. PLACSP — Plataforma de Contratación del Sector Público (Spain)

2. TED — Tenders Electronic Daily (EU)

FTS — Find a Tender Service (United Kingdom)

4. TenderNed (Netherlands)

5. BOAMP (France)

6. Doffin (Norway)

7. Sell2Wales (Wales, United Kingdom)

8. Public Contracts Scotland (Scotland, United Kingdom)

9. Simap (Switzerland)

10. IMO (International Maritime Organization)

Fase 2 — Roadmap (verified sources, not yet ingested)

Estas fuentes están verificadas y operativas pero aún no integradas al pipeline automático de TenderWatch. Requieren browser automation (Playwright/nodriver) o registro oficial. Sprint separado tras validar el round-robin Tier A.

🇬🇧 Reino Unido (devolved + sector público)

🇪🇺 UE + EEE + Suiza

Para una guía paso-a-paso de cómo registrarse como proveedor en cada uno de estos portales, ver Cómo registrarse como proveedor.

Methodology

Collection pipeline

1. Fetch: Cron-triggered scrapers download source feeds (HTTP GET, no authentication).
2. Parse: XML/JSON parsed into canonical Tender records (source, source_id, title, description, buyer, value, dates, URL).
3. PII redaction: Before any text is processed by an LLM, PII is stripped (emails, phones, names, expediente IDs). Redacted text is never shown to users — only LLM summaries are.
4. LLM summarisation: Each tender is summarised in Spanish (peninsular strict) and British English (en-GB). Summaries are stored separately and tagged by source.
5. Deduplication: Same tender across sources is linked via normalised source_id; canonical record is kept.
6. Index: Indexed for fast search (MySQL FULLTEXT on title + description).

Update cadence

• PLACSP: Every hour. ~50–200 new tenders per run.
• TED: Once per business day (Mon–Fri), 07:00 UTC.
• FTS: Once per business day (Mon–Fri), 08:00 UTC. Intermittent due to upstream throttling.

Validation

Every ingested tender is validated against the following checks:

• URL reachable: Source URL must return HTTP 200 at scrape time.
• Required fields: source_id, title, buyer_country, source_url must be present and non-empty.
• Title length: Between 10 and 500 characters (anything outside this range is rejected or cleaned).
• Date sanity: deadline_date must be after publication_date and not more than 5 years in the future.
• Value sanity: estimated_value ≥ 0 and < €10 billion (anti-typo).

Error reporting

If you find an error in a tender (wrong title, broken source URL, outdated data):

• Email: tickets@libertydreams.co.uk
• Include the tender URL and a description of the issue.
• We aim to investigate within 2 business days and correct the source pipeline (not just the display).

Licensing and reuse

All data ingested by TenderWatch is sourced from public-sector feeds published under national and EU transparency legislation. TenderWatch holds no proprietary rights over the underlying data; we provide value through aggregation, multilingual summarisation, and search.

Reuse by third parties: If you wish to reuse TenderWatch data (e.g. for journalism, research, or another service), please contact tickets@libertydreams.co.uk. Bulk export is available to enterprise customers via API.

What TenderWatch extracts vs what it does not

For each scraped tender, TenderWatch extracts only the descriptive and administrative fields of the notice: official title, technical description of the contract object, contracting body name, country, CPV code, procedure type, estimated value, publication date, submission deadline, official language, canonical URL of the source notice, and the full raw payload (in the internal raw field, kept for traceability).

TenderWatch does NOT extract direct identification or contact fields of the procurement officer (contact-point name, personal email, direct telephone, fax). These fields stay at the public source and never travel to our database, nor to the AI provider we use to generate summaries.

Empirical analysis (sample of 260 TED notices + 6 full XMLs, October 2026): 0.0% of examined tenders carry identifiable PII within scraped fields. This property is preserved by an automated regression test (tests/test_pii_regression.py) which asserts that no scraper promotes PII fields into the Tender model.

Historial del catálogo de fuentes

• 2026-09-03: Source catalogue documented; methodology page first published.
• 2026-09-01: PII redaction pipeline verified; LLM summary language enforcement (Spanish peninsular strict, British English en-GB).
• 2026-08-25: PLACSP parser fix for Atom feed (corrected corruption of expediente IDs in description field).
• 2026-08-15: PLACSP, TED, FTS scrapers deployed.

Nota: TenderWatch no mantiene un changelog público de producto (decisión §13). Este historial documenta únicamente cambios al catálogo de fuentes scrapeadas; las novedades operativas se reportan solo internamente al equipo.