{"id":4406,"date":"2023-09-06T07:40:37","date_gmt":"2023-09-06T07:40:37","guid":{"rendered":"https:\/\/test.opensource-db.in\/wp1\/?p=4406"},"modified":"2023-09-06T08:11:30","modified_gmt":"2023-09-06T08:11:30","slug":"data-engineering-with-hydra-the-basics","status":"publish","type":"post","link":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/","title":{"rendered":"Data Engineering with Hydra &#8211; The Basics"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">It is already well-known that Postgres itself offers a solid foundation for the efficient and speedy execution of analytical processes, and we have chosen Hydra to enhance these features further. It uses advanced techniques like columnar storage, vectorized execution, and query parallelization to handle queries efficiently. With these features, Hydra significantly improves the performance of analytical workloads in Postgres.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Columnar storage:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;Columnar storage in Postgres refers to a technique where data is organized and stored column by column instead of row by row. Columnar tables support updates, deletes, and vacuuming. Hydra leverages columnar storage, and the data organization in Hydra is often referred to as \u201c<strong>Stripes<\/strong>\u201d and \u201c<strong>Chunks<\/strong>\u201d.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Stripes:<\/strong> A stripe typically contains a certain number of rows from each column The number of rows in a stripe can vary depending on the implementation and configuration of the columnar storage system.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Chunks: <\/strong>The data is further divided into smaller units called &#8220;chunks&#8221;. A chunk contains a fixed number of contiguous values from a single column. The purpose of dividing data into chunks is to enable efficient compression and data retrieval.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This design choice has several advantages for analytical workloads:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Multifaceted: <\/strong>Columnar storage is often considered while working with huge volumes of data. HTAP(Hybrid Transactional\/Analytical Processing) enables the storage of metadata and does real-time data analytics. Columnar storage is efficient for loading new data quickly.&nbsp;<\/li><li><strong>Data Compression: <\/strong>Columnar storage is well-suited for data compression. Similar values within a column can be efficiently encoded, reducing storage requirements and faster data access.<\/li><li><strong>Aggregation Efficiency:<\/strong> Analytical queries often involve aggregating data, and columnar storage excels in performing these types of operations, resulting in faster query execution.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Our columnar approach:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;Download the relevant Linux columnar package. Initialize the Postgres data cluster, add all necessary dependencies, and configure it as necessary.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>git clone git@github.com:rbernierZulu\/columnar-tables-extension.git\nsudo dpkg -i postgresql-15-citus-columnar_11.1-1-UBUNTU2004_amd64.deb\nsudo apt-get update<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"811\" height=\"151\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\" alt=\"\" class=\"wp-image-4407\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png 811w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10-300x56.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10-768x143.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10-532x99.png 532w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10-430x80.png 430w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10-394x73.png 394w\" sizes=\"(max-width: 811px) 100vw, 811px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Vectorized Execution:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;A technique where operations are performed on entire columns at once, rather than processing rows individually. This offers several benefits:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Increased Processing Speed: <\/strong>By operating on entire columns in a vectorized manner, Hydra takes advantage of modern processor architectures optimized for parallelism and SIMD (single instruction, multiple data) operations. This leads to substantial performance improvements.<\/li><li><strong>Reduced CPU Overhead:<\/strong> Vectorized execution reduces the need for repetitive CPU instructions, resulting in lower CPU overhead and better utilization of computational resources.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Query Parallelization:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;Hydra efficiently processes queries in parallel, distributing the workload across multiple processors to speed up the analytical processing.<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Enhanced Scalability: <\/strong>Parallel query processing allows Hydra to scale horizontally by adding more processing nodes. As query volume and complexity increase, the system can handle load efficiently without compromising on performance.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Support Heap Tables:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;Heap tables are the default table types in Postgres. If we want a traditional row-based table, In Hydra we need to use \u2018USING heap\u2019 at the end of a CREATE TABLE statement. Then they would store data row-wise, making them well-suited for transactional workloads. Hydra&#8217;s integration with heap tables brings several advantages:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Transactional Workloads:<\/strong> In scenarios where real-time data processing is required, such as capturing and processing continuous data streams, Heap tables provide the necessary efficiency for handling frequent inserts, updates, and deletes.<\/li><li><strong>Optimal Data Ingestion: <\/strong>Heap tables excel in rapidly ingesting data due to their efficient row-wise storage. They are thus well-suited for real-time data-driven applications involving a constant stream of incoming data or bulk data uploads.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Unveiling the data mystery: <\/strong>Seamless loading of postgresql data into a Hydra data warehouse<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In previous sections, we delved into the intricacies of data optimization, columnar storage, vectorized execution, and the capabilities that Hydra brings to the table. Now let&#8217;s take a step further and explore the practical steps involved in seamlessly loading postgresql data into the Hydra data warehouse.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Buckle up as we embark on a journey to harness the power of Hydra for unmatched analytical performance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Setting up:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;We start off with an operational Postgresql database, which serves as the source. Next, we&#8217;ll ensure that the Hydra data warehouse is likewise operational &#8211; setting it up only takes a few minutes and simple actions.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"779\" height=\"470\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15.png\" alt=\"\" class=\"wp-image-4408\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15.png 779w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15-300x181.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15-768x463.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15-164x99.png 164w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15-133x80.png 133w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-19-15-394x238.png 394w\" sizes=\"(max-width: 779px) 100vw, 779px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"463\" height=\"474\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58.png\" alt=\"\" class=\"wp-image-4409\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58.png 463w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58-293x300.png 293w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58-97x99.png 97w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58-78x80.png 78w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-13-20-58-296x303.png 296w\" sizes=\"(max-width: 463px) 100vw, 463px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">To add a team, a warehouse, a region, and a capacity, we only need to log in to Hydra and set their values. The storage and computational resources can be chosen as per our needs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is the actual data present in the database before taking a dump. This is the data we are going to restore to Hydra.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"67\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1024x67.png\" alt=\"\" class=\"wp-image-4411\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1024x67.png 1024w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-300x20.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-768x50.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1536x100.png 1536w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1515x99.png 1515w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1224x80.png 1224w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-394x26.png 394w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-915x60.png 915w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19-1240x81.png 1240w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-12-22-19.png 1852w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>LOAD DATA:<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">First, we must have the PostgreSQL source database.&nbsp; Here we are using <strong>pg_dump<\/strong> and <strong>pg_restore<\/strong> to transfer the load onto Hydra.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pg_dump:<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;By using pg_dump we can capture data from specific tables into the database by using <strong>&nbsp;-t tablename1<\/strong>&nbsp; <strong>-t tablename2.<\/strong> Similarly, while importing data onto Hydra, it will create columnar tables by default.&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pg_dump --no acl -Fc --no-owner -h localhost -U postgres -d db_1 &gt; db_1.dump<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"242\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-1024x242.png\" alt=\"\" class=\"wp-image-4412\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-1024x242.png 1024w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-300x71.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-768x182.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-418x99.png 418w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-338x80.png 338w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-394x93.png 394w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-915x217.png 915w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48-1240x294.png 1240w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-04-12-09-48.png 1318w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Once the dump is created for the existing data, the very next step is to create connectivity between the PostgreSQL database and Hydra. For that, we add details about our Hydra setup to the pg_service.conf service file.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"875\" height=\"200\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08.png\" alt=\"\" class=\"wp-image-4413\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08.png 875w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08-300x69.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08-768x176.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08-433x99.png 433w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08-350x80.png 350w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-46-08-394x90.png 394w\" sizes=\"(max-width: 875px) 100vw, 875px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The database dump that we created earlier using pg_dump is now imported into Hydra using pg_restore by passing the hostname, username, password, and database name of the Hydra database (which we can find on the dashboard) as arguments.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>    Pgpassword=&lt;postgres_password&gt; \\\n     pg_restore \\\n    -h hydra-hostname.us-east-1.hydras.io \\\n    -p port \\\n    -U username \\\n    -d database name \\\n     Mydb.dump<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"167\" src=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-1024x167.png\" alt=\"\" class=\"wp-image-4414\" srcset=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-1024x167.png 1024w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-300x49.png 300w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-768x125.png 768w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-1536x251.png 1536w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-607x99.png 607w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-490x80.png 490w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-394x64.png 394w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-915x149.png 915w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04-1240x202.png 1240w, https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-47-04.png 1851w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here is the data that has been restored to the Hydra Data warehouse.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We will explore more in future sessions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>It is already well-known that Postgres itself offers a solid foundation for the efficient and speedy execution of analytical processes, [&hellip;]<\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[23,42],"tags":[81,80,85],"class_list":["post-4406","post","type-post","status-publish","format-standard","hentry","category-postgresql-14","category-postgresql-15","tag-data-engineering","tag-data-warehousing","tag-hydra"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Data Engineering with Hydra - The Basics - OpenSource DB<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Engineering with Hydra - The Basics - OpenSource DB\" \/>\n<meta property=\"og:description\" content=\"It is already well-known that Postgres itself offers a solid foundation for the efficient and speedy execution of analytical processes, [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\" \/>\n<meta property=\"og:site_name\" content=\"OpenSource DB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-09-06T07:40:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-06T08:11:30+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\" \/>\n<meta name=\"author\" content=\"Brahmini Ratnam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@opensource_db\" \/>\n<meta name=\"twitter:site\" content=\"@opensource_db\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Brahmini Ratnam\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\"},\"author\":{\"name\":\"Brahmini Ratnam\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301\"},\"headline\":\"Data Engineering with Hydra &#8211; The Basics\",\"datePublished\":\"2023-09-06T07:40:37+00:00\",\"dateModified\":\"2023-09-06T08:11:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\"},\"wordCount\":925,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\",\"keywords\":[\"data-engineering\",\"data-warehousing\",\"hydra\"],\"articleSection\":[\"PostgreSQL 14\",\"PostgreSQL 15\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\",\"name\":\"Data Engineering with Hydra - The Basics - OpenSource DB\",\"isPartOf\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\",\"datePublished\":\"2023-09-06T07:40:37+00:00\",\"dateModified\":\"2023-09-06T08:11:30+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\",\"contentUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png\",\"width\":811,\"height\":151},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/test.opensource-db.in\/wp1\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Engineering with Hydra &#8211; The Basics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#website\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/\",\"name\":\"OpenSource DB\",\"description\":\"Your Trusted OpenSource Databases partner\",\"publisher\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/test.opensource-db.in\/wp1\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\",\"name\":\"OPENSOURCE DB PRIVATE LIMITED\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png\",\"contentUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png\",\"width\":368,\"height\":120,\"caption\":\"OPENSOURCE DB PRIVATE LIMITED\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/\",\"https:\/\/x.com\/opensource_db\",\"https:\/\/www.youtube.com\/channel\/UCmTI5h\",\"https:\/\/www.linkedin.com\/company\/opensource-db\",\"https:\/\/www.instagram.com\/opensource_db\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301\",\"name\":\"Brahmini Ratnam\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g\",\"caption\":\"Brahmini Ratnam\"},\"url\":\"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Engineering with Hydra - The Basics - OpenSource DB","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Data Engineering with Hydra - The Basics - OpenSource DB","og_description":"It is already well-known that Postgres itself offers a solid foundation for the efficient and speedy execution of analytical processes, [&hellip;]","og_url":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/","og_site_name":"OpenSource DB","article_publisher":"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/","article_published_time":"2023-09-06T07:40:37+00:00","article_modified_time":"2023-09-06T08:11:30+00:00","og_image":[{"url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png","type":"","width":"","height":""}],"author":"Brahmini Ratnam","twitter_card":"summary_large_image","twitter_creator":"@opensource_db","twitter_site":"@opensource_db","twitter_misc":{"Written by":"Brahmini Ratnam","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#article","isPartOf":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/"},"author":{"name":"Brahmini Ratnam","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301"},"headline":"Data Engineering with Hydra &#8211; The Basics","datePublished":"2023-09-06T07:40:37+00:00","dateModified":"2023-09-06T08:11:30+00:00","mainEntityOfPage":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/"},"wordCount":925,"commentCount":0,"publisher":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#organization"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage"},"thumbnailUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png","keywords":["data-engineering","data-warehousing","hydra"],"articleSection":["PostgreSQL 14","PostgreSQL 15"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/","url":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/","name":"Data Engineering with Hydra - The Basics - OpenSource DB","isPartOf":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#website"},"primaryImageOfPage":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage"},"thumbnailUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png","datePublished":"2023-09-06T07:40:37+00:00","dateModified":"2023-09-06T08:11:30+00:00","breadcrumb":{"@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#primaryimage","url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png","contentUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2023\/09\/Screenshot-from-2023-09-06-11-14-10.png","width":811,"height":151},{"@type":"BreadcrumbList","@id":"https:\/\/test.opensource-db.in\/wp1\/data-engineering-with-hydra-the-basics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/test.opensource-db.in\/wp1\/"},{"@type":"ListItem","position":2,"name":"Data Engineering with Hydra &#8211; The Basics"}]},{"@type":"WebSite","@id":"https:\/\/test.opensource-db.in\/wp1\/#website","url":"https:\/\/test.opensource-db.in\/wp1\/","name":"OpenSource DB","description":"Your Trusted OpenSource Databases partner","publisher":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/test.opensource-db.in\/wp1\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/test.opensource-db.in\/wp1\/#organization","name":"OPENSOURCE DB PRIVATE LIMITED","url":"https:\/\/test.opensource-db.in\/wp1\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/","url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png","contentUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png","width":368,"height":120,"caption":"OPENSOURCE DB PRIVATE LIMITED"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/","https:\/\/x.com\/opensource_db","https:\/\/www.youtube.com\/channel\/UCmTI5h","https:\/\/www.linkedin.com\/company\/opensource-db","https:\/\/www.instagram.com\/opensource_db\/"]},{"@type":"Person","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301","name":"Brahmini Ratnam","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g","caption":"Brahmini Ratnam"},"url":"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/"}]}},"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"Brahmini Ratnam","author_link":"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/test.opensource-db.in\/wp1\/category\/postgres\/postgresql-14\/\" rel=\"category tag\">PostgreSQL 14<\/a> <a href=\"https:\/\/test.opensource-db.in\/wp1\/category\/postgres\/postgresql-15\/\" rel=\"category tag\">PostgreSQL 15<\/a>","rttpg_excerpt":"It is already well-known that Postgres itself offers a solid foundation for the efficient and speedy execution of analytical processes, [&hellip;]","_links":{"self":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/4406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/comments?post=4406"}],"version-history":[{"count":6,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/4406\/revisions"}],"predecessor-version":[{"id":4421,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/4406\/revisions\/4421"}],"wp:attachment":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/media?parent=4406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/categories?post=4406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/tags?post=4406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}