Lakehouse Platform with AI Speed Layer Part of the Model Driven Unified & Governed Real-Time Data Platform Suite (GUM-RTDP)— Source-code licensed reference architecture

What It Does

The Lakehouse Platform transforms raw business events into trusted, governed analytical assets — in near real time. Built on a medallion architecture (Bronze → Silver → Gold), it delivers clean dimensional data to dashboards and APIs with the response times your business demands. It also embeds an AI Speed Layer that scores business events as they occur, enabling fraud detection, anomaly identification, and risk assessment in real-time.

Medallion Architecture

  • Bronze Zone — raw event ingestion from Kafka topics into MinIO object storage via incremental micro-batch processing. Immutable, auditable, replayable.
  • Silver Zone — business entity integration layer built on the CEDM (Conceptual Enterprise Data Model). Raw events are transformed into clean, governed entities — customer, product, order — with temporal tracking preserving full history.
  • Gold Zone — dimensional consumption layer built on the DEDM (Dimensional Enterprise Data Model). Star schema ready for BI dashboards and analytical APIs with sub-second response times via an in-process serving layer.

Serving Layer

  • The Gold Zone (and sometimes the silver zone) is partly replicated in the serving layer which is essentially a low latency DB built for speed which is used by dashboards and API.

    The reference architecture is using an in-process analytical engine that reads Delta tables directly from object storage and creates a replicated DB with views delivering dashboard and API response times. This architecture is tool-agnostic — it does not impose BI or low latency DB such as Snowflake, Synapse, Tableau, or Power BI, preserving your freedom of choice.

AI Speed Layer

The AI Speed Layer is the real-time intelligence engine embedded within the Lakehouse Platform. Its business objectives:

  • Fraud Detection — identify fraudulent order patterns before they are processed, not hours later
  • Anomaly Detection — flag statistically abnormal transactions that fall outside expected business behavior in real time

These are supplied as examples, that are contextual to an order system, there are many more AI models that could be implemented in similar way, for more details on those please refer to the Industry Use Case.

Inference is performed using ONNX-based models (Random Forest for fraud scoring, Isolation Forest for anomaly detection) running in a dedicated Go-based inference engine. Results are published back to the event fabric as first-class business events — consumed by the Order Management application for threshold evaluation and human review escalation.

End-to-end latency: under 1.5 seconds (as low as 0.8 seconds in Direct EDA Mode) from event to threshold processing of scored result.

For context — a major Canadian bank flagged a fraudulent transaction 24 hours after it occurred. Your organization can do it in under 1.5 seconds.

Technology Foundation

Apache Kafka · Apache Spark · Delta Lake · MinIO · ONNX · Go · Python — each proven individually at scale by Tier 1 technology organizations. The Lakehouse Platform is the only reference architecture that integrates all of them into a governed, real-time lakehouse with an embedded AI Speed Layer.

The Lakehouse Platform is part of the GUM-RTDP suite alongside the EDA Platform and Data Governance Platform. Source-code licensed for mid-market and enterprise deployment.

→ Contact Us to discuss your Lakehouse deployment