The layer everything else
turns out to depend on
Every stalled AI initiative we are called into traces back to the same place: data that cannot be reached safely, at quality, by the people who need it. Fixing that is unglamorous and it is the work that unblocks everything downstream.
You probably recognise at least three of these
Four numbers, one metric
Finance, operations and the board each have their own definition of the same figure, and the monthly meeting is spent reconciling them rather than acting on them.
Overnight batch you have outgrown
Decisions made on yesterday's picture because the pipeline runs once a night and nobody wants to touch it.
A warehouse only two people can change
Business logic accumulated across a decade of stored procedures, undocumented, with the original authors long departed.
Reports that break silently
An upstream schema change lands on a Tuesday and nobody notices until a customer asks why a number looks wrong.
No answer to "where did this come from?"
The question every auditor and every sceptical executive asks, and the one nobody can answer without a week of investigation.
AI blocked on data access
Every AI proposal dies at the privacy review because there is no governed, de-identified copy of anything.
A lakehouse, built in layers
Bronze, silver, gold — with contracts between them, so a change upstream fails loudly at the boundary instead of quietly in a report.
Ingestion, batch and streaming
Change data capture from operational databases, event streams from applications, file drops from partners and telemetry from the field — landed once, in raw form, with the original preserved.
- DMS and Kinesis for CDC and streaming
- Amazon MSK where Kafka semantics are required
- Idempotent, replayable landing by design
Storage on S3 with Apache Iceberg
Open table format, so schema evolution does not require a rebuild and your data is not hostage to one query engine's proprietary storage.
- Schema evolution without downstream breakage
- Time travel for reproducible analysis
- Multiple engines over one copy of the data
Transformation with contracts
Business logic in version control, tested like application code, with explicit contracts between layers. A breaking change fails the pipeline rather than the quarterly report.
- Data contracts enforced at layer boundaries
- Quality tests with owner-routed alerting
- One definition per metric, in one place
Governance and access
A catalogue with named custodians, lineage from source to published product, and access granted by purpose rather than by ticket.
- Lake Formation and Glue Data Catalog
- Column and row-level access controls
- De-identification for analytical and AI use
Useful in eight weeks, not eighteen months
We build one vertical slice end to end before broadening. A platform with three domains that people actually use beats a complete architecture nobody has adopted.
See client engagementsWeeks 1–2 · Domain selection
Pick the domain where a governed dataset unblocks the most decisions. Usually not the biggest one — the one with the most waiting consumers.
Weeks 3–6 · Vertical slice
Ingestion through to a published gold dataset, with contracts, quality tests, lineage and access controls. Real users querying it by week six.
Weeks 7–10 · Platform hardening
The reusable machinery: ingestion patterns, deployment pipelines, catalogue conventions and the runbook your team will operate from.
Ongoing · Domain onboarding
Each subsequent domain gets faster. By the fourth, your team is onboarding them without us — which is the actual success measure.
What we build with
Storage & format
S3 · Apache Iceberg · Lake Formation · Glue Data Catalog
Ingestion
DMS · Kinesis · Amazon MSK · AppFlow · Glue
Processing & query
EMR Serverless · Athena · Redshift · dbt · Spark
Orchestration
Step Functions · MWAA · EventBridge · Lambda
Start with one domain that matters
Ten weeks to a governed dataset people query daily, plus the platform machinery to onboard the next one in a fortnight.