I’m working at a 50-something company that is just starting to grow enough to the point that is warranting a dedicated Data Analyst and an Engineer for our internal systems. This means that we are looking into our current solution and looking for ways to improve and support the predict growth.
Until recently, the volume and usage patterns didn’t warrant a “proper” scalable system so we were getting by with having all our data sources consolidated (via Stitch) to a single PostgreSQL database as our destination, with Redash to connect and visualization.
As our demands grow, it feels like the “natural” path would be to take this data out of PostgreSQL and move it to a “proper” Data Warehouse like Redshift or Snowflake. At the same time, I really like the flexibility and peace of mind that I have by not being locked in to any of the solutions provided by the Big Tech.
So, I’d like to know if anyone has been worked with the open source, self-hostable data warehouse? I’ve looked into Apache Druid and Hydra, and it seems that they could be reasonable alternatives. What would you recommend?