Skip to main content

9 posts tagged with "Druid"

View All Tags

Reduce Apache Druid™ cloud costs with storage tiering and query routing

· 11 min read
Calum Miller
Director

1. Match your cloud expenditure to the work your data does​

Apache Druid™ clusters can become expensive when years of rarely queried history occupy the same infrastructure as frequently accessed dashboard data. Long-running analytical queries can add pressure to that infrastructure, encouraging teams to buy more capacity for every workload.

Millersoft helps companies reduce cloud expenditure by profiling data access, configuring hot, warm and cold storage, and selecting suitable query engines for different workloads. The aim is to protect the performance users need while reducing capacity that brings little business value.

Our approach connects storage placement, query execution and actual cloud billing. Moving data off premium disks creates a saving only when the deployment can release or resize the corresponding resources.

Apache Druid™ data repair: profiling and deep storage recovery

· 11 min read
Calum Miller
Director

1. When the cluster is running but the data cannot be trusted​

Incorrect Apache Druid™ data can remain hidden behind a healthy dashboard. A replayed batch can inflate revenue, a transformation can put events in the wrong day, or a damaged segment can leave part of the historical record unavailable.

Millersoft helps investigate and correct these problems by combining data profiling, segment-level diagnosis and reconciliation to trusted sources. We identify the affected datasources and time ranges, establish the cause, and plan a repair that preserves valid records and can be verified afterwards.

Our experience with large Apache Druid™ datasets, ingestion pipelines, customised deployments and migration reconciliation provides the foundation for this work. The objective is to restore an explainable analytical result and address the fault that produced it.

Apache Druid™ upgrade assistance and migration services

· 9 min read
Calum Miller
Director

1. Executive summary​

Upgrading open source Apache Druid™ means protecting the analytical service your business already depends on. Streaming ingestion, historical data, custom extensions and business-critical dashboards all need to work together after the change.

Millersoft provides specialist Apache Druid™ upgrade assistance, including assessments of older installations, incremental upgrade planning, large-volume data migration and reconciliation between environments. Our experience also covers high-availability deployments on AWS, Kubernetes operations and maintaining customised open source codebases.

Whether you need to bring an ageing cluster forward or plan routine maintenance of an established platform, we can help define the upgrade path, rehearse it and validate the result.

Apache Superset™ Enhanced Email Capabilities for CC, BCC and body

· One min read
Aidan Mulgrew
Software Engineer

In early versions Superset’s email reporting capabilities were relatively limited, typically supporting only basic recipient and subject fields.

Millersoft expanded this functionality significantly by adding CC and BCC fields for broader communication flexibilit. Superset now supports a custom email body, allowing teams to tailor their message content beyond the standard “Explore in Superset” link.

Enterprise ready Apache Superset™ authentication with Keycloak

· One min read
Aidan Mulgrew
Software Engineer

Millersoft has integrated Keycloak authentication, enabling teams to manage user access through a central identity provider. This integration supports enterprise requirements like single sign-on (SSO), role-based access control (RBAC), and multi-factor authentication. Enabling organisations to manage Superset users securely and at scale.


Read the full post

Unlocking the Power of Druid MSQ in Superset

· 3 min read
Aidan Mulgrew
Software Engineer

At Millersoft, we’ve been working to make Apache Druid™ more efficient, flexible, and cost-effective for analytics workloads. One area we’ve focused on is the Python-to-Druid connector (pydruid), which powers integrations with Apache Superset™.

We’re excited to share some improvements we’ve made that open the door to new capabilities in Superset dashboards and the potential for substantial cost savings in large-scale Druid deployments.

Counting Distinct Values Accurately in Druid SQL

· 3 min read
Aidan Mulgrew
Software Engineer

When working with Druid SQL, it's easy to fall into a common trap when counting distinct values: using COUNT(DISTINCT ...) directly can sometimes return unexpected results. Recently, I hit a case where COUNT(DISTINCT) returned a different value than selecting the DISTINCT rows manually - and this post explains why that happens, and how to fix it.