Scope and Requirements: The scope and requirements for building a Data Warehouse (DWH) involve many factors. Let’s look at them one by one. First comes the volume of data, understanding that the size and complexity of DWH depend on the … Read More
Author Archives: Brahmini Ratnam
PostgreSQL performance tuning from Manual to Automatic – DBtune
Introduction: In the previous blog post – The DNA of DBA: Mastering Parameter Tuning, we explored how mastering PostgreSQL parameter tuning is essential for every successful DBA, and how leveraging DBtune’s AI-driven insights can automate optimization. Now, let’s delve deeper … Read More
Streaming PostgreSQL Data to kafka Topics using Debezium
Introduction: In today’s fast-paced digital landscape, real-time data processing is no longer a luxury but a necessity. Businesses need to react to changes instantly, whether for updating inventories, personalising user experiences, or detecting fraud. This need for immediacy has led … Read More
PostgreSQL OID Datatype: Impacts and Remedies
Introduction PostgreSQL – The World’s Most Advanced Open Source Relational Database, introduced a significant change with version 12: the discontinuation of the Object Identifier (OID) datatype. This move has left many developers and database administrators pondering over its implications and … Read More
Postgres Materialized Views: A Practical 101 – Part 2
Introduction Welcome back to the second installment of our deep dive into the Materialized views! In our previous post, we looked at the basics of materialized views, showcasing how they can help query performance and simplify complex data aggregations with … Read More
Postgres Materialized Views: A Practical 101 – Part 1
Introduction In the realm of database management, efficiency and speed are king. PostgreSQL: The World’s Most Advanced Open Source Relational Database, offers a feature that is nothing short of magical for data analysts and developers alike: Materialized Views. Unlike standard … Read More
Performance tuning with Apache Spark – Introduction
Introduction: Welcome back to our ongoing series on Data transformation with Apache Spark! In our previous posts, we’ve covered essential topics like setting up Apache Spark on Ubuntu, integrating data with Spark, and querying datasets using Apache Drill. Now, we’re … Read More
Open-source Data Engineering with PostgreSQL
Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3 INTRODUCTION: Welcome back! Following our exploration of data movement between PostgreSQL and Amazon S3 in the previous blog, we now venture into the realm of querying with Apache Drill. In … Read More
Open-source Data Engineering with PostgreSQL
Blog-3: Data Loading with Apache Spark INTRODUCTION: Welcome to the next installment of our series on Open-source Data Engineering with PostgreSQL. In this blog, we’ll delve into the practicalities of transforming table data from PostgreSQL into the Parquet format and … Read More
Open-source Data Engineering with PostgreSQL
Blog-2: Installation and Setup on Ubuntu INTRODUCTION: Welcome back to the series on Open-source Data Engineering with PostgreSQL. In this post, we shall delve into the installation and configuration of Apache Spark and Apache Drill on an Ubuntu environment. Our … Read More