04 / 01 / 2024
KEYONE Data
Large-scale analytics engine
- Role
- Year
- Links
- Announcement
Overview
A project built specifically for large-scale analytics, handling hundreds of millions of records.
Approach
DuckDB and Apache Parquet as the storage and execution layer.
A FastAPI query layer built on top of it.
The engineering focus was performance, pagination and query optimisation.
Stack
- DuckDB
- Apache Parquet
- FastAPI
- Python
- Query Optimisation
- Pagination