Open Source OLAP databases for HealthTech

HealthTech Landscape

HealthTech is a rapidly growing industry that is transforming the way healthcare is delivered. One of the key technologies that is enabling this transformation is OLAP databases. OLAP databases are designed to support complex analytical queries on large volumes of data. This makes them ideal for a variety of HealthTech applications, such as:

• Clinical decision support: OLAP databases can be used to develop clinical decision support systems that help clinicians make more informed decisions about patient care.
• Population health management: OLAP databases can be used to develop population health management systems that help healthcare providers track and manage the health of their patient populations.
• Pharmaceutical research and development: OLAP databases can be used to develop pharmaceutical research and development systems that help pharmaceutical companies develop new drugs and treatments.

StarRocks is an open source query engine that that delivers data warehouse performance on the data lake.

Query data on top of the lake, support performant JOINS at scale, support 1000s of users doing adhoc queries

Run StarRocks on top of raw data and then create views or materialized views as needed.

AirBnB with StarRocks: 4 JOINS with billions of rows in under 4 seconds

Tencent Games with StarRocks: 400+ users doing ad hoc queries on xx+ petabytes of data on Apache Iceberg files.

StarRocks can help in the following ways:

• Data Silo and Data Integration: Healthcare data is often siloed in different systems, making it difficult to get a complete view of a patient’s health. OLAP databases can help to break down these silos and provide a unified view of patient data. However, this requires that the data be integrated from different systems, which can be a complex and challenging task. StarRocks helps solve this issue by using the MySQL wire protocol. Since the MySQL wire protocol is well-documented and widely supported, there are many libraries and tools available to help with the integration. In addition, StarRocks can execute all the various JOIN types and supports JSON and other data formats which reduces the data pipeline engineering and simplifies the data stack (no need for complex data denormalization or complex data transformations).
• Performance: OLAP databases need to be able to handle complex analytical queries on large volumes of data. StarRocks was designed to provide sub-second query response time for GB and PB amounts of data.
• User Expectations: Users of analytics system expect the system to be fast because the users’ time is expensive. Moving queries from minutes to seconds to even sub-seconds allow for the medical staff to be more productive.
• Open Standards: StarRocks can use open table formats like Apache Hive, Delta Lake, Apache Iceberg and Apache Hudi. This allows you to store you data in a open standards way so that you won’t be locked in by a vendor.
• Costs: By using object storage, you can reduce the costs of your data storage costs up to 90%. Even in situations where there is hardware parity, StarRocks gives you better performance due to it’s vectorized engine, multi-level cache and query cost based optimizer.

Read more about StarRocks at http://starrocks.io

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. StarRocks received InfoWorld’s 2023 BOSSIE Award for best open source software.