Job Description

Project Role Description: We are looking for a Cloudera DWH Analyst with strong experience in building Data Warehouses on Cloudera Data Platform (CDP) using Hive and Spark. The candidate will be responsible for designing and implementing data warehouse layers using Medallion Architecture (Bronze, Silver, Gold) and building scalable data pipelines and transformation workflows. The role requires hands-on expertise in Hive, Spark, SQL, and big data warehousing concepts, along with experience working in distributed data platforms.

Key Responsibilities:

Data Warehouse Design

• Design and implement Data Warehouse solutions on Cloudera Data Platform (CDP).
• Build data warehouse layers using Medallion Architecture (Bronze, Silver, Gold layers).
• Design data models and schema structures for analytics and reporting.

Data Pipeline Development

• Develop and manage data ingestion and transformation pipelines using Apache Spark and Hive.
• Implement ETL/ELT workflows to move data from source systems into the data warehouse layers.
• Ensure efficient data processing and transformation for analytics workloads.

SQL Development & Data Processing

• Write and optimize complex SQL queries on large datasets.
• Develop Hive queries and Spark jobs for batch data processing.
• Optimize queries and storage structures for performance and scalability.

Data Integration

• Integrate data from multiple sources such as databases, files, APIs, and enterprise systems.
• Implement data cleansing, transformation, and validation processes.

Performance Optimization

• Optimize Hive tables, partitioning strategies, and storage formats.
• Improve performance of Spark jobs and distributed data processing tasks.

Collaboration & Documentation

• Work closely with data engineers, architects, analysts, and business stakeholders.
• Document data models, pipeline designs, and transformation logic.
• Ensure adherence to data governance and best practices.

Qualifications:

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or related field.
• 4–7 years of experience in Data Warehousing and Big Data environments.

Required Skills:

• Strong experience with Cloudera Data Platform (CDP).
• Hands-on experience with Hive and Spark.
• Experience designing Data Warehouse architectures using Medallion Architecture.
• Strong SQL development and query optimization skills.
• Experience building data pipelines and ETL workflows.
• Understanding of distributed data processing and big data concepts.

Preferred Skills:

• Experience with Airflow or other workflow orchestration tools.
• Familiarity with Impala for high-performance querying.
• Experience with data governance tools such as Ranger and Atlas.
• Knowledge of data lakehouse architectures and analytics platforms.