Jefrey Li
Taipei
Summary
Experienced in data engineering, data warehousing, and data modeling, specializing in optimizing Spark ETL and building end-to-end pipelines for batch processing and CDC. Skilled in driving efficient data processing, storage, and retrieval, with a focus on resolving performance bottlenecks for seamless integration. Dedicated to delivering solutions that enable data-driven decision-making.
Overview
9
9
years of professional experience
1
1
Certification
Work History
Senior Data Architect
Agoda
Bangkok, Thailand
11.2023 - 04.2024
- Optimized the Spark ETL based in Hadoop system to do cost reduction, especially in refactoring Spark SQL code, Spark cluster configuration fine-tuning, and data warehousing architecture.
- Provided technical guidance on Spark cluster configuration and SQL optimization strategies to the team, enabling members to independently optimize their ETL jobs.
Data Engineer
JKOPay Co. Ltd.
Taipei, Taiwan
05.2022 - 10.2023
- Dockerized a Spark standalone cluster to run ETL, and used NiFi to do incremental data sync.
- According to business logics to do data warehousing, data modeling, and transformation of ODS (Operational Data Store) from various data sources into data marts using columnar data warehouse technics (DML, DDL, Materialized Views, Live Views, etc.)
- To ensure data correctness and consistency, developing the change data capture (CDC) system using Debezium to capture any row-level DML operations from databases, and then the data is aggregated into data warehouses.
Data Engineer
iKala Interactive Media Inc.
Taipei, Taiwan
11.2021 - 03.2022
- Built system integration testing to ensure data correctness
Data Engineer
OneAD Inc.
Taipei, Taiwan
11.2018 - 10.2021
- Designed a web application using Django with Docker, enabling users to create custom segment requests based on customer needs, with the data processing handled by Spark.
- Integrated third-party vendors’ data with OneAD’s data and utilized RESTful APIs to transmit the processed audience data to each demand-side platform.
- Developed a web application allowing users to query and download AD campaign reports, where the aggregated data was processed through either ClickHouse queries or Spark jobs running on GCP Dataproc.
Mechanical Engineer
Ship and Ocean Industries R&D Center
New Taipei City, Taiwan
05.2016 - 10.2018
- R&D alternative service
Equipment Engineer
Taiwan Semiconductor Manufacturing Company
Hsinchu County, Taiwan
10.2015 - 05.2016
- R&D alternative service
Education
Master of Science - Graduate Institute of Applied Mechanics
National Taiwan University
Taipei, Taiwan08.2015
Bachelor of Science - Department of Mechanical Engineering
National Chiao Tung University
Hsinchu County, Taiwan06.2013
Skills
Programming: Proficient in Python for data engineering and automation
Big Data Batch Processing: Expertise in Spark for large-scale processing and analytics for historical records
CDC & ETL: Experience with Debezium integrated Kafka for real-time data ingestion and dbt for data transformation
Data Warehousing & Modeling: Strong in data warehouse architecture design, and data modeling for high data quality service
Query Optimization: Advanced SQL skills, with experience in ClickHouse for high-performance and queries
Containerization: Skilled in Docker for application deployment in Linux environments
Collaboration & Team Leadership: Worked closely with stakeholders to align data strategies with business goals, and contributed to the expansion and upskilling of the data analytics team
Languages
Mandarin
Proficient
C2
English
Upper intermediate
B2
Certification
- [Area of expertise] License - [Timeframe]
Timeline
Senior Data Architect
Agoda
11.2023 - 04.2024
Data Engineer
JKOPay Co. Ltd.
05.2022 - 10.2023
Data Engineer
iKala Interactive Media Inc.
11.2021 - 03.2022
Data Engineer
OneAD Inc.
11.2018 - 10.2021
Mechanical Engineer
Ship and Ocean Industries R&D Center
05.2016 - 10.2018
Equipment Engineer
Taiwan Semiconductor Manufacturing Company
10.2015 - 05.2016
Master of Science - Graduate Institute of Applied Mechanics
National Taiwan University
Bachelor of Science - Department of Mechanical Engineering
National Chiao Tung University
Similar Profiles
Omar AbdelkhalekOmar Abdelkhalek
Senior Software Engineer at AgodaSenior Software Engineer at Agoda
Ayyappan SampathAyyappan Sampath
Director of Engineering at AgodaDirector of Engineering at Agoda
VIONA TRINATHANAELVIONA TRINATHANAEL
Customer Experience Specialist at AgodaCustomer Experience Specialist at Agoda