Krishna P.

A seasoned professional specializing in the design and architecture of ETL pipelines: 1. On GCP, adept at leveraging Apache Beam with Java on Dataflow/Dataprep, Python, and Composer for seamless workflow orchestration. 2. On AWS, proficient in utilizing Apache Spark with Scala/Python on AWS Glue/EMR clusters. Additionally, skilled in employing MWAA (Amazon Managed Workflows for Apache Airflow) for streamlined workflow orchestration. I specialize in delivering sophisticated products characterized by high-quality code, primarily in Scala, Java, and Python. Leveraging Apache Spark, I design solutions to efficiently extract data from diverse sources such as MySQL, MongoDB, Elasticsearch, DynamoDB, and more. This data is then ingested into a Data Lake using Hudi or DeltaLake table formats, stored on platforms like S3 or GCS, and made discoverable and accessible through a catalog with Hive integration. Subsequently, I facilitate further ingestion from the Data Lake into enterprise Data Warehouses such as BigQuery, Redshift, or other data warehousing solutions. I've also gained experience in crafting microservices with Java and Spring Boot, utilizing databases such as MySQL, MongoDB, and Elasticsearch. The resulting web application can be deployed on either our server or your own. It is designed to be robust, ensuring high availability, and free from bugs. Additionally, it supports NGINX gateway setup and facilitates the discovery of microservices. At the heart of my expertise lies the ability to collaborate closely with clients, tailoring solutions to meet their specific needs. I provide comprehensive services under a single umbrella, ensuring affordability without compromising on quality. My commitment is to establish enduring relationships with clients built on trust and satisfaction. As a data architect and developer, I specialize in creating robust, high-availability, and scalable web applications with a focus on achieving high throughputs. If you have any inquiries or require assistance, please feel free to reach out to me. I am here to help! I have recently earned the Professional Data Engineer certification from Google Cloud. You can verify my credentials at [login to view URL].