Our client is a global leader in digital transformation and data-driven innovation, currently expanding their global data team to support a wide range of digital initiatives.

They are now seeking a Data Engineer to join their Data Service team and play a key role in building scalable data pipelines and analytics solutions.

Key Responsibilities:

* Design, build, and optimize scalable end-to-end data pipelines for structured and unstructured data from diverse sources (APIs, databases, files).
* Develop and manage robust ETL/ELT workflows for both batch and real-time data processing, ensuring high data quality and reliability.
* Monitor and optimize pipeline performance using autoscaling, caching, and cost-effective strategies.
* Leverage Databricks and Apache Spark (PySpark, Spark SQL) for distributed data processing and transformation.
* Implement data storage solutions using Medallion Architecture and Delta Lake, ensuring schema evolution and ACID compliance.
* Collaborate with data scientists, analysts, engineers, and business stakeholders to align data solutions with business goals.
* Stay current with emerging technologies in data engineering, especially within the Azure and Databricks ecosystem.

Key Requirements:

* Bachelor's degree in Computer Science, Information Systems, or a related field.
* 5-10 years of hands-on experience in data engineering, with a strong track record in building and optimizing data pipelines.
* Proven expertise in Databricks, Apache Spark (PySpark, Spark SQL), Python, and SQL.
* Experience integrating Databricks with Azure services (Data Lake Storage, Data Factory, SQL Database, Event Hubs).
* Familiarity with both batch and real-time data processing tools such as Azure Event Hubs and Kafka.
* Proficient in Git and Azure DevOps for CI/CD and source control, including Databricks Asset Bundles.
* Strong understanding of data quality, monitoring, and performance optimization.
* Excellent documentation, collaboration, and communication skills.
* Mature, independent, and customer-oriented with strong time management and problem-solving abilities.

Our client offers competitive compensation and benefits, as well as a dynamic and fast-paced work environment.

Interested parties, please send your CV to Ron Lee on [email protected] or contact on +852 39725817 for more details. For other available opportunities, please visit www.connectedgroup.com

Employer Ref.: AVCM-134883

Please apply through the Employer.