Skip to main content

← All solutions

Reduce Apache Druid™ cloud costs with storage tiering and query routing

· 11 min read
Calum Miller
Director

1. Match your cloud expenditure to the work your data does​

Apache Druid™ clusters can become expensive when years of rarely queried history occupy the same infrastructure as frequently accessed dashboard data. Long-running analytical queries can add pressure to that infrastructure, encouraging teams to buy more capacity for every workload.

Millersoft helps companies reduce cloud expenditure by profiling data access, configuring hot, warm and cold storage, and selecting suitable query engines for different workloads. The aim is to protect the performance users need while reducing capacity that brings little business value.

Our approach connects storage placement, query execution and actual cloud billing. Moving data off premium disks creates a saving only when the deployment can release or resize the corresponding resources.

Millersoft Apache Druid™ cost optimisation: an illustrative policy places days 0 to 30 on fast hot storage, days 31 to 180 on economical warm Historicals, and older history in queryable cold deep storage. Interactive dashboards use the native engine and loaded segments. Suitable heavy historical reports use asynchronous MSQ tasks reading deep storage. Profile access, benchmark latency and concurrency, then resize resources and verify the bill. Illustrative monthly costs of £24,000 fall to £18,400 after including extra task compute and storage access costs; these are assumptions, not cloud prices or a client result. (opens the full-size image in a new tab)

2. Start with workload and cost profiling​

Age alone does not determine whether data is cold. An older accounting period may be queried intensively during an audit, while recent raw events may be rarely accessed after a summary has been produced.

We examine query logs and metrics to establish which datasources, time ranges and business applications consume resources. Useful measures include query frequency, scanned intervals, latency, concurrency, CPU, memory pressure, disk throughput, ingestion lag and task duration.

We connect those findings to cloud billing: Historical compute and attached disks, ingestion and query workers, Brokers, object storage, requests, data transfer, metadata services and monitoring. Savings are evaluated against an agreed latency and availability target.

A cost assessment can reveal several different opportunities:

  • Rarely accessed segments occupying premium local storage and replicated Historical capacity.
  • Overprovisioned disk performance or instance sizes for the measured workload.
  • Heavy reports competing with interactive dashboards for shared resources.
  • Query workers left running when their scheduled workloads have finished.
  • Unbounded dashboard queries repeatedly scanning far more history than users need.

The output is a workload map and an evidence-based proposal, with expected trade-offs and a baseline against which to measure the change.

3. Configure hot, warm and cold storage deliberately​

Hot, warm and cold are useful design labels. They describe access and performance requirements; they are not three automatic disk modes that Apache Druid™ enables with one setting.

Hot: fast storage for frequent, time-sensitive queries​

Hot data typically supports operational dashboards, customer-facing analytics and recent-event investigation. We assess fast local NVMe or appropriately provisioned SSD-backed storage, together with the CPU, memory and network capacity needed to use it effectively.

The hot tier needs sufficient replicas and spare capacity for the agreed failure scenarios. We tune it for measured concurrency and response-time requirements rather than choose the largest instance by default.

Warm: economical Historical capacity for regular analysis​

Warm data is queried less frequently or has a more relaxed latency target, but still benefits from being loaded on Historical processes. It may fit lower-cost SSD configurations, different instance shapes or less expensive disk options where benchmarks show acceptable performance.

We assess disk capacity, IOPS, throughput and segment-loading time together. A cheap disk that makes routine queries or recovery unacceptably slow is a poor saving.

Apache Druid™ supports Historical tiers and rules governing segment placement and replica counts. We use those controls to express the agreed access policy. See Apache's load-rule documentation.

Cold: retain queryable history in deep storage​

For suitable infrequent workloads, older segments can remain in deep storage without a continuously loaded Historical copy. The savings opportunity is to reduce local serving capacity while retaining an appropriate path to the history.

The query path has prerequisites: datasource schema must be available, and the selected execution engine must support the required access. We validate schema discovery and backfill requirements before unloading historical ranges. See Apache's deep-storage querying guidance.

Cold queryable storage also needs an appropriate cloud storage class. For example, Amazon S3 Glacier Flexible Retrieval and Deep Archive require restoration before access. We review lifecycle policies so required segment objects remain accessible within the agreed response time. See AWS's archive restoration documentation.

Keep placement, retention and recovery separate​

We review rule order, inherited defaults and retention behaviour before applying changes. Segments intended for deep-storage queries must remain used in metadata; marking them unused can remove them from that query path. Kill tasks and object lifecycle deletion policies need a separate retention decision.

This makes unloading a serving copy a controlled capacity change, with a tested reload or alternative query route for exceptional demand.

4. Move heavy queries to an appropriate execution path​

Different workloads need different combinations of startup time, concurrency, data access and execution cost. We assess the options against representative queries and the installed Apache Druid™ release.

WorkloadCandidate execution pathWhat we assess
Frequent interactive dashboardsNative engine with loaded hot or warm segmentsResponse time and concurrency under realistic demand
Long historical scans, exports and scheduled reportsMulti-stage query (MSQ) task engineStartup overhead, worker capacity, source freshness and total job cost
Large joins or high-cardinality exact aggregationsDart, where suitable for the installed releaseExperimental status, feature compatibility and Historical resource consumption
Broader lakehouse analysis across multiple systemsAn external engine over a compatible lake datasetData preparation, freshness, duplicate storage and whether Druid™ work is genuinely displaced

Native queries for interactive work​

We preserve a predictable path for the dashboards that need low latency. Query time filters, sensible result limits and workload-aware connections help keep routine exploration bounded.

Query laning and service tiering can also help manage contention within the native query workload. They are resource controls within a deployment, rather than a switch to another engine. See Apache's mixed-workload guidance.

MSQ task queries for suitable heavy reports​

Apache Druid™ provides an asynchronous query path through the MSQ task engine that can read deep-storage segments. It suits work that can accept a different response-time profile, such as historical analysis and scheduled extracts. See Apache's query-from-deep-storage API.

Millersoft can integrate that path into supported applications, including reviewing the Apache Superset™ connector and reporting workflow. Submission, status polling, result retrieval, cancellation and timeout handling must work together. Changing a connection label alone does not establish that a query is running through MSQ.

For task-based historical reports, we agree a published-data cut-off and check whether it satisfies the reporting requirement before substituting that path for a live streaming view.

MSQ query tasks consume worker capacity. If they share resources with ingestion or maintenance jobs, they can still cause contention. We assess worker pools, concurrency and scheduling, and establish isolation where the deployment supports it. Worker startup, intermediate storage, object requests and data transfer belong in the cost comparison.

Dart for workloads that benefit from its execution model​

Dart is another option to evaluate for complex analytical queries. Apache currently documents it as experimental and describes execution on Brokers and Historicals, including support for large joins and exact high-cardinality aggregations. See Apache's Dart documentation.

Dart can change performance characteristics, but it still consumes Historical compute. We benchmark it separately from task-based MSQ and check connector compatibility and resource controls before proposing it for a production workload.

External engines when a separate data path makes sense​

Some companies already maintain an authoritative lake dataset alongside Apache Druid™. Heavy cross-system analysis may be better placed there, with the serving cluster reserved for time-sensitive analytics.

We can assess an external engine such as Trino against a compatible representation, for example an Iceberg table. That assessment includes building and maintaining the representation, reconciling results and agreeing its refresh delay. See Trino's Iceberg connector.

A Trino connection to Druid™ is a different arrangement: the connector can push work down to Druid™. It does not by itself prove that the Historical workload has been removed. See Trino's Druid™ connector and pushdown behaviour.

5. An illustrative storage and query policy​

For a workload where recent data dominates interactive access, an initial hypothesis might be:

Data agePlacementTypical access
Most recent 30 daysHot Historicals on fast disksFrequent dashboards and operational investigation
Days 31 to 180Warm Historicals on benchmarked economical storageRegular reporting with a more relaxed latency target
Older than 180 daysQueryable deep storageScheduled historical reports through MSQ tasks

These ranges are illustrative. We select actual boundaries from access patterns, including reporting deadlines and seasonal demand. A regularly queried older period may remain warm or be temporarily promoted.

A report spanning recent and older data must use a tested path with complete coverage. We verify results across tier boundaries, schema changes, late-arriving records and the freshness guarantees of each engine. Users should not receive a plausible partial result because their connection can see only loaded intervals.

6. Measure the net saving, including shifted costs​

A useful business case includes the cost of the replacement query path. Consider this illustrative monthly model, using assumed figures rather than cloud price quotations or a client result:

Cost categoryBeforeProposed steady state
Historical compute and attached disks£18,000£10,000
Ingestion and MSQ task compute£1,000£3,000
Shared platform and monitoring£3,000£3,000
Object storage, requests and data transfer£2,000£2,400
Total£24,000£18,400

The model produces a £5,600 monthly reduction, approximately 23.3%, after allowing for additional task compute and storage access. Delivery costs and temporary parallel capacity are assessed separately to establish payback.

The result depends on releasing billable resources. Unloading segments while retaining the same provisioned disks and instances may leave the bill largely unchanged. Reserved capacity, commitments, attached-volume sizing and peak requirements can also affect realised savings.

We compare equivalent workloads, performance targets and resilience before and after the change. Useful measures include cost per reporting run, cost per retained period, dashboard latency, ingestion freshness and the time needed to restore capacity for an exceptional query period.

7. Millersoft's experience and delivery approach​

Millersoft's Apache Druid™ services cover ingestion and query tuning, high-availability AWS deployments, Kubernetes operations and large-volume migration with reconciliation.

Our published MSQ and Apache Superset™ integration work describes extending the pydruid connector for MSQ execution and query cancellation. That experience helps connect infrastructure changes with the reporting tools people actually use.

Cloud cost and workload assessment
Profile access patterns and resource consumption, reconcile them with billing, and identify the capacity a revised design could release.
Hot, warm and cold storage design
Assess disk performance, Historical tiers, replica policies and deep-storage queryability against workload and recovery requirements.
Query routing and integration
Evaluate native, MSQ task and other suitable execution paths, then test application behaviour and resource isolation.
Measured implementation and support
Pilot one workload, validate coverage and performance, resize resources in stages, and monitor the realised saving.

We start with one datasource or reporting workload. After benchmarking the proposed configuration, we implement the placement and routing changes, validate the results, and reduce capacity only when the evidence supports it. Recovery checkpoints protect the ability to restore the previous serving arrangement.

Further reading​

Spend on the performance your users need

Talk to Millersoft about reducing your Apache Druid™ cloud expenditure. We can help profile the workload, evaluate storage tiers and query engines, and build a measured plan for releasing unnecessary capacity.

Discuss a Druid™ cost assessment

Apache Druid™ and Apache Superset™ are trademarks of the Apache Software Foundation. Millersoft is not affiliated with the Apache Software Foundation.