{"id":5390,"date":"2024-01-31T06:13:51","date_gmt":"2024-01-31T06:13:51","guid":{"rendered":"https:\/\/test.opensource-db.in\/wp1\/?p=5390"},"modified":"2024-01-31T06:54:31","modified_gmt":"2024-01-31T06:54:31","slug":"open-source-data-engineering-with-postgresql-4","status":"publish","type":"post","link":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/","title":{"rendered":"Open-source Data Engineering with PostgreSQL"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">INTRODUCTION:<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Welcome back! Following our exploration of data movement between PostgreSQL and Amazon S3 in the previous blog, we now venture into the realm of querying with Apache Drill. In this sequel, we\u2019ll demonstrate the simplicity of querying PostgreSQL tables, local Parquet files, and Parquet files on S3. Join us as we seamlessly connect diverse data sources using the power of Apache Drill and SQL. Let\u2019s continue our data journey.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Configuring the Apache Drill for PostgreSQL<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To begin querying PostgreSQL data, the first step is to add the PostgreSQL storage plugin.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Open Apache drill web UI using <a href=\"http:\/\/localhost:8047\/storage\" target=\"_blank\" rel=\"noreferrer noopener\">http:\/\/localhost:8047\/storage<\/a>. Here we need to add this code as a PostgreSQL plugin providing the necessary details such as database URL, username, and password. Save the configuration, now Apache Drill can recognize PostgreSQL as a data source.<\/li>\n\n\n\n<li>An alternate approach is Apache Drill, which also supports adding storage plugins through CLI(command line interface. we can add the plugin to <code>storage-plugins-override.conf<\/code> file.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"type\": \"jdbc\",\n  \"enabled\": true,\n  \"driver\": \"org.postgresql.Driver\",\n  \"url\": \"jdbc:postgresql:\/\/localhost:5432\/spark_db\",\n  \"username\": \"postgres\",\n  \"password\": \"postgres\",\n  \"sourceParameters\": {\n    \"minimumIdle\": 0,\n    \"autoCommit\": true,\n    \"connectionTestQuery\": \"select version() as postgresql_version\",\n    \"dataSource.cachePrepStmts\": true,\n    \"dataSource.prepStmtCacheSize\": 250\n  }\n\n}<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><img fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/lh7-us.googleusercontent.com\/aSWRwGxn7e5vbcTcyXAM3mUir5dzWM09puyhJhowqzbVL_H6h0jqdhUWmEfi0Nd45Cvkd_Ndt7DIAnfqOJiiSLaJEwSy-XM5Vm2ReH1qfhaNi6aSTWnIWF4Sj6ikmkgQWU4CuZv5X9xs6IIci2L4uhA\" width=\"501\" height=\"450\"><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Now we are ready to explore SQL power across PostgreSQL, We can visualize the PostgreSQL across Drill using commands such as <code>SHOW SCHEMAS<\/code>;&nbsp; <code>SHOW DATABASES<\/code>;<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" width=\"583\" height=\"443\" src=\"https:\/\/lh7-us.googleusercontent.com\/nDDPb9GOw3PjwEYlV10OqBR54JIhJG-aUYWKVwH05QLXr3jcJKIDIepx-9EupZdPe5I2NNUY34vPAboIsAkZTIOyOnrY7M2wWNNiwVfv_sCHoHD4nzsGkC2PRsfrG0fQSrVQj4iTeSnuHKpmLk7u_hI\"><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We can use the specific schema as per our need (example \u2018public\u2019) using the command <code>\u2018USE postgresql.public<\/code>\u2019, then we can see the tables present in that database using the \u2018<code>SHOW TABLES<\/code>\u2019 command<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><img decoding=\"async\" width=\"602\" height=\"136\" src=\"https:\/\/lh7-us.googleusercontent.com\/edpOKnnJZ93c3iSNS4w1ft9x1C7SsSGGlwBGF-nTCL4hsABi4YTIDVTp0oCDteC5gckWe1cAMmwRoYbWRtIz0fSy1GH8H2Y5Fqu9NKMuDxDE7orem5oY97lZP1ugvfyXNisViLVX32qLS3hZJ0q6bYo\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"188\" src=\"https:\/\/lh7-us.googleusercontent.com\/B8KiIA2c-dwmP1_n2pfv0lrMwk2V1jQnegBwCe4FLujlIMrGPkVQi4rQDGbhoYHA4PeapD84rlY9e5SPUW7ZytGzJpFv1oZIZ1m7VUDh-tQGehc18yaJU8LORas6E8yrqlzbIXKl5frvSKCBmVflgN8\"><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We can now run the SQL query to read the data in the desired table<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"120\" src=\"https:\/\/lh7-us.googleusercontent.com\/bLwafYl0O7sN05hzHsjUFleuG-M2e-3kI6lh0I-XBtgiLf6ca6NMLVTzvde24nueRVXB0OnyoT2yKIvVGmgcW9aiCo9LyxA6s1DgSY2eK1G5xINxC4aBf5c_0ia0zVlMHa-DxZp4WNOc514pZTn4lLM\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Working on Local Parquet Files:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Now, let\u2019s move on to querying Parquet files stored locally<\/li>\n\n\n\n<li>For querying parquet files we don&#8217;t need any specific plugins to be added as it has parquet file support inbuilt<\/li>\n\n\n\n<li>The beauty of Drill is that it allows direct querying for extracting data from parquet format and displaying the query output in a&nbsp; readable format<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"201\" src=\"https:\/\/lh7-us.googleusercontent.com\/6RR6EL7lay2sWW6_SMfv7uj_yrSdRr7TAnMOn5r4AwCUSNJR6D1IH3s9fOUv4KjKnefqw1QUY7mWNBdM2ZovqAd0AAzVyAvRcq7QAlZ7RHNhH2zJBsUxX0hsqgxG4yOEw5SAtopr7ZKqOCghDGqPPRw\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Working on S3 parquet Files:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>As we previously mentioned the Parquet files reading doesn\u2019t need any specific configuration; and Drill seamlessly interacts with S3 as well, but a few things have to be updated in the existing configuration file like the AWS Access Key ID, the Secret Key, and the bucket name for Drill to be able to access data stored in Amazon S3.&nbsp;<\/li>\n\n\n\n<li>Here is the query for extracting data from Parquet files in the S3 bucket<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><img loading=\"lazy\" decoding=\"async\" width=\"602\" height=\"205\" src=\"https:\/\/lh7-us.googleusercontent.com\/yaAGYxiLMGt6LhduwLWSAfO0d71zqb7PkaToT4zBjAIxkthk60QHsNVE-RdXsmJl7kBbSeowcG3zgV6ajCMCMSnKufzK9ud7RGtJQOG0azx3g1SZjlzYjDc80ZIbj5PmOI8_J3nc9M7Bo0TsHgW6uco\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Notice the simplicity &#8211; no complex configurations are needed. We can effortlessly translate the data to formats with SQL queries using Apache Drill.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Upcoming??<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In this blog, we delved into the magic of Apache Drill, starting our journey from configuring PostgreSQL storage plugins to minimal configuration updates for S3 interactions, we highlighted the simplicity of querying the PostgreSQL, local Parquet files, and S3-residing Parquet files.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Stay tuned for the final installment in our series! The upcoming blog focuses on optimizing performance using Apache Spark and Apache Drill. Get ready to elevate your data engineering game with insights into enhancing query speed and maximizing efficiency. Until then, happy querying and exploring.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3 INTRODUCTION: Welcome back! Following our exploration of data movement between [&hellip;]<\/p>\n","protected":false},"author":10,"featured_media":5402,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[23,42,89],"tags":[108,81,26],"class_list":["post-5390","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-postgresql-14","category-postgresql-15","category-postgresql-16","tag-apache-drill","tag-data-engineering","tag-postgresql"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Open-source Data Engineering with PostgreSQL - OpenSource DB<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Open-source Data Engineering with PostgreSQL - OpenSource DB\" \/>\n<meta property=\"og:description\" content=\"Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3 INTRODUCTION: Welcome back! Following our exploration of data movement between [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\" \/>\n<meta property=\"og:site_name\" content=\"OpenSource DB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-31T06:13:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-31T06:54:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1800\" \/>\n\t<meta property=\"og:image:height\" content=\"945\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Brahmini Ratnam\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@opensource_db\" \/>\n<meta name=\"twitter:site\" content=\"@opensource_db\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Brahmini Ratnam\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\"},\"author\":{\"name\":\"Brahmini Ratnam\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301\"},\"headline\":\"Open-source Data Engineering with PostgreSQL\",\"datePublished\":\"2024-01-31T06:13:51+00:00\",\"dateModified\":\"2024-01-31T06:54:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\"},\"wordCount\":495,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png\",\"keywords\":[\"apache drill\",\"data-engineering\",\"postgresql\"],\"articleSection\":[\"PostgreSQL 14\",\"PostgreSQL 15\",\"PostgreSQL 16\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\",\"name\":\"Open-source Data Engineering with PostgreSQL - OpenSource DB\",\"isPartOf\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png\",\"datePublished\":\"2024-01-31T06:13:51+00:00\",\"dateModified\":\"2024-01-31T06:54:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png\",\"contentUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png\",\"width\":1800,\"height\":945},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/test.opensource-db.in\/wp1\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Open-source Data Engineering with PostgreSQL\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#website\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/\",\"name\":\"OpenSource DB\",\"description\":\"Your Trusted OpenSource Databases partner\",\"publisher\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/test.opensource-db.in\/wp1\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#organization\",\"name\":\"OPENSOURCE DB PRIVATE LIMITED\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png\",\"contentUrl\":\"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png\",\"width\":368,\"height\":120,\"caption\":\"OPENSOURCE DB PRIVATE LIMITED\"},\"image\":{\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/\",\"https:\/\/x.com\/opensource_db\",\"https:\/\/www.youtube.com\/channel\/UCmTI5h\",\"https:\/\/www.linkedin.com\/company\/opensource-db\",\"https:\/\/www.instagram.com\/opensource_db\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301\",\"name\":\"Brahmini Ratnam\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g\",\"caption\":\"Brahmini Ratnam\"},\"url\":\"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Open-source Data Engineering with PostgreSQL - OpenSource DB","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Open-source Data Engineering with PostgreSQL - OpenSource DB","og_description":"Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3 INTRODUCTION: Welcome back! Following our exploration of data movement between [&hellip;]","og_url":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/","og_site_name":"OpenSource DB","article_publisher":"https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/","article_published_time":"2024-01-31T06:13:51+00:00","article_modified_time":"2024-01-31T06:54:31+00:00","og_image":[{"width":1800,"height":945,"url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png","type":"image\/png"}],"author":"Brahmini Ratnam","twitter_card":"summary_large_image","twitter_creator":"@opensource_db","twitter_site":"@opensource_db","twitter_misc":{"Written by":"Brahmini Ratnam","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#article","isPartOf":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/"},"author":{"name":"Brahmini Ratnam","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301"},"headline":"Open-source Data Engineering with PostgreSQL","datePublished":"2024-01-31T06:13:51+00:00","dateModified":"2024-01-31T06:54:31+00:00","mainEntityOfPage":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/"},"wordCount":495,"commentCount":0,"publisher":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#organization"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage"},"thumbnailUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png","keywords":["apache drill","data-engineering","postgresql"],"articleSection":["PostgreSQL 14","PostgreSQL 15","PostgreSQL 16"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/","url":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/","name":"Open-source Data Engineering with PostgreSQL - OpenSource DB","isPartOf":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#website"},"primaryImageOfPage":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage"},"thumbnailUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png","datePublished":"2024-01-31T06:13:51+00:00","dateModified":"2024-01-31T06:54:31+00:00","breadcrumb":{"@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#primaryimage","url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png","contentUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png","width":1800,"height":945},{"@type":"BreadcrumbList","@id":"https:\/\/test.opensource-db.in\/wp1\/open-source-data-engineering-with-postgresql-4\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/test.opensource-db.in\/wp1\/"},{"@type":"ListItem","position":2,"name":"Open-source Data Engineering with PostgreSQL"}]},{"@type":"WebSite","@id":"https:\/\/test.opensource-db.in\/wp1\/#website","url":"https:\/\/test.opensource-db.in\/wp1\/","name":"OpenSource DB","description":"Your Trusted OpenSource Databases partner","publisher":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/test.opensource-db.in\/wp1\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/test.opensource-db.in\/wp1\/#organization","name":"OPENSOURCE DB PRIVATE LIMITED","url":"https:\/\/test.opensource-db.in\/wp1\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/","url":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png","contentUrl":"https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2021\/10\/osdb-logo-tm-2.png","width":368,"height":120,"caption":"OPENSOURCE DB PRIVATE LIMITED"},"image":{"@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/people\/OpenSource-DB\/100072970755470\/","https:\/\/x.com\/opensource_db","https:\/\/www.youtube.com\/channel\/UCmTI5h","https:\/\/www.linkedin.com\/company\/opensource-db","https:\/\/www.instagram.com\/opensource_db\/"]},{"@type":"Person","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/867a56696ebca29a93489ed53609f301","name":"Brahmini Ratnam","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/test.opensource-db.in\/wp1\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/05bcc95d3352ddae501c8cdae6b20b8e4e59c373cb7e506042427bca7fe13a3b?s=96&d=mm&r=g","caption":"Brahmini Ratnam"},"url":"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/"}]}},"rttpg_featured_image_url":{"full":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",1800,945,false],"landscape":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",1800,945,false],"portraits":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",1800,945,false],"thumbnail":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4-150x150.png",150,150,true],"medium":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4-300x158.png",300,158,true],"large":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4-1024x538.png",1024,538,true],"1536x1536":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4-1536x806.png",1536,806,true],"2048x2048":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",1800,945,false],"ultp_layout_landscape_large":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",1200,630,false],"ultp_layout_landscape":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",870,457,false],"ultp_layout_portrait":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",600,315,false],"ultp_layout_square":["https:\/\/test.opensource-db.in\/wp1\/wp-content\/uploads\/2024\/01\/image-4.png",600,315,false]},"rttpg_author":{"display_name":"Brahmini Ratnam","author_link":"https:\/\/test.opensource-db.in\/wp1\/author\/brahmini-ratnam-meruva\/"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/test.opensource-db.in\/wp1\/category\/postgres\/postgresql-14\/\" rel=\"category tag\">PostgreSQL 14<\/a> <a href=\"https:\/\/test.opensource-db.in\/wp1\/category\/postgres\/postgresql-15\/\" rel=\"category tag\">PostgreSQL 15<\/a> <a href=\"https:\/\/test.opensource-db.in\/wp1\/category\/postgres\/postgresql-16\/\" rel=\"category tag\">PostgreSQL 16<\/a>","rttpg_excerpt":"Blog-4: Apache Drill Magic across PostgreSQL, Local Parquet, and S3 INTRODUCTION: Welcome back! Following our exploration of data movement between [&hellip;]","_links":{"self":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/5390","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/comments?post=5390"}],"version-history":[{"count":6,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/5390\/revisions"}],"predecessor-version":[{"id":5407,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/posts\/5390\/revisions\/5407"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/media\/5402"}],"wp:attachment":[{"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/media?parent=5390"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/categories?post=5390"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/test.opensource-db.in\/wp1\/wp-json\/wp\/v2\/tags?post=5390"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}