Cloud Data Pipelines Engineer

• Programming: Advanced proficiency in Python and/or R, with strong experience writing clean, modular, and well-tested code.
• Cloud Platforms: Hands-on production experience with either Google Cloud Platform (GCP) or Microsoft Azure.
• Containerization & CI/CD: Practical experience with Docker, Kubernetes, and CI/CD pipelines (GitHub Actions, GitLab CI).
• Data Stores: Proficiency with SQL and relational/non-relational data warehouses (PostgreSQL, BigQuery, Snowflake, or Azure SQL).
• Software Engineering Foundations: Solid grasp of version control (Git), automated testing practices, and API design principles.

,[R/Python Workload Execution: Containerize, deploy, and orchestrate custom R and Python scripts to run reliably in cloud-native serverless or containerized environments., Cloud Integration: Leverage managed cloud services on GCP (Cloud Functions, Cloud Run, BigQuery, Dataflow) or Azure (Azure Data Factory, Azure Functions, Databricks, Synapse Analytics)., Workflow Orchestration: Implement and maintain workflow automation tools (such as Apache Airflow or Prefect) to schedule and monitor script execution., Data Quality & Reliability: Implement automated logging, error handling, and testing frameworks to validate data integrity and script outputs., Performance Optimization: Tune data processing jobs and cloud resource allocation for cost-efficiency, execution speed, and scalability.] Requirements: Python, R, Cloud platform, Google Cloud Platform, GCP, Microsoft Azure, Docker, Kubernetes, CI/CD Pipelines, GitHub Actions, GitLab CI, SQL, Data warehouses, PostgreSQL, BigQuery, Snowflake, Azure SQL, Git, Automated testing, API