Faster queries on average
Across the research firm’s analytical queries.
We built an AWS data platform for a market and financial research firm: connecting external sources, preparing linked records in ClickHouse, and giving analysts a workspace to query and explore them.
Swipe across the demo to explore.
Ask in plain language. Read the SQL. Run it, explore the answer and add a chart to a dashboard.
Outcomes from the
client’s project account.
Across the research firm’s analytical queries.
By the same analytics team.
The firm attributes this growth to faster, more accurate analysis.
From more than 50 sources, 550 TB stored.
Revenue attribution reflects the client’s assessment of faster, more accurate analysis.
Different formats. Different schedules. The same company under different names. Analysts spent time collecting and reconciling records before they could research a market.
We connected the S3 lake, databases, registries, APIs and scrapers through streaming and scheduled ingestion.
Spark cleans, deduplicates and links records before they enter the ClickHouse analytical store.
Readable SQL, plain-language questions, dashboards, notebooks and search sit on the prepared data.
A question runs against data already in the analytical store. Ingestion and preparation happen before the analyst asks.
>50sources
S3 data lake, company databases, public registries, APIs and scrapers.
Kafkastreaming
Airflowscheduled loads
~15Mnew records / day
Spark cleans and deduplicates records, links the same company across sources, and prunes unused data.
Shared preparation before the research question.
550 TBstored
Prepared, linked data in one analytical store.
SQL, dashboards, notebooks, search and plain-language-to-SQL.
Read, check and change the SQL before running it.
Built on AWS. Sources keep their own delivery schedules; analysts query the prepared store.
Inspect a query, trace a company’s sources, explore a market or check a feed. Four views of the same connected platform.
Swipe across the demo to explore.
The workbench shows the schema beside existing SQL. An analyst inspects the query, changes ORDER BY and runs it again.
The results reorder and the chart exposes detail. Plain-language SQL generation is shown in the opening tour.
Cleaning and entity linking happen in the pipeline. Analysts query records already prepared for use together.
Kafka streams and Airflow loads feed the same processing path. Each source keeps its own delivery cadence.
Pruning unused data was part of the platform, alongside source status and pipeline monitoring.
Four months of development and one month of support for this project, from connecting the sources to helping analysts move their work over.
Sources and ingestion
Processing
Store and tools
Search and launch
We stayed with the team as analysts moved their work onto the platform, helping with the switch and tuning the parts they found slow.
Analysts moved queries and reporting into the connected workspace.