Principal Data Engineer (AWS, SQL, Data Architecture)

JOB PURPOSE:

As a Senior Data Engineering, you will be responsible for:

- Analyze statistics, data, and algorithms for projection accuracy for the deployed data model.

- Create, maintain, and design relational databases to support enterprise application and physical data modelling according to project requirements for data acquisition and security as well as customer-defined deliverables.

- Develop strategies for data modelling, design, and implementation to meet stated requirements for Metadata management, Operation Data Stores, Data Lake, Data Warehouse, Data Marts and Extract Transform Load environments in batch processing, and real-time injection.

- Review existing data architectures to determine overall effectiveness and compliance with determined objectives, develop comprehensive strategies for improving or replacing underperforming areas, and come up with plans to execute these solutions.

- Create and test physical data models for a variety of business data, applications, database structures, and metadata tables to meet operational goals for performance and efficiency.

- Carrying out day-to-day support for the reporting system

MAJOR ACCOUNTABILITIES:

- Take the ownership of designing, building and managing data pipeline on both batch processing and streaming to ensure its availability, reliability and usability for various business demands.

- Review the existing data workflow and model to provide possible solutions for cost optimization, performance and efficiency

Other assignments such as incidents, ad-hoc tasks, etc..

TECHNICAL SKILLS:

- Strong Experienced in Data Streaming Workflow by using AWS Components such as AWS Glue Job (PySpark), Glue Data Catalogs, Glue Data Quality, Lake Information, Lambda, S3, IAM

- Strong Experienced in application data architecture, data design patterns such as Data Lake, Data Warehouse for Life Insurance

- Strong knowledge of SQL and NO SQL Data such as Spark query, Athena, Redshift, Postgres, SQL Server, DynamoDB, and In Memory Database.

- Good knowledge in ETL Batch, Micro Batch workflow by using SSIS

- Good knowledge of life cycle of application developments.

- Good knowledge in DevOps tools and practices (source control, build, CI…).

SOFT SKILLS:

- Effective project communication

- Collaborative Problem-Solving session

- Critical evaluation of Data Architecture

- Embracing change and learning

- Cross-function team projects

PROBLEM SOLVING:

- Optimized the existing data workflow to prevent recurring issues, leading to improved stability and performance

- Strong analytical and problem-solving skills with a track record of delivery effective and sustainable solutions.

COMMUNICATION:

Collaborated cross-functionally to troubleshoot and resolve intricate data discrepancies, enhancing data integrity