LME Group is the world centre findustrial metals trading clearing. Most of the world’s non-ferrous metals business is conducted on the LME totalling $15.7 trillion, 185 million lots 4.1 billion tonnes in 2018.
The metals community uses the LME, a member of HKEX Group, as a venue to transfer take on price risk, as a physical market of last resort as the provider of transparent global reference prices.
Overall Purpose of Role:
This role is a critical part of the Enterprise data team involved in the replacement of legacy ETL (Informatica) tool by providing key data engineering activities including pipeline management, analysis & visualisation engineering. The role will be working closely alongside ETL developers wider technology teams to engineer solutions supporting their strategic roadmap.
This is a high-impact role fa candidate who is passionate about engineering excellence, have a strong technical background excellent IT skills paired with excellent team working communication skills. This role is a data engineering role (covering backend, data, infrastructure) collaborates on solution design, implementation, deployment, testing support.
//Responsibilities/
• Design, implement, maintain robust data pipelines infrastructure to support LME integration across data warehouses critical to ensure reliability scalability.
• Ensure the robustness quality of data workloads using Python/Java/Scala modern data engineering practices, including automated validation, monitoring, comprehensive testing.
• Ensure all technical documentation is accurate, up-to-date, accessible to relevant stakeholders.
• Provide internal data analysis reporting to support business technology objectives.
• Act as a liaison between technical teams non-technical stakeholders, ensuring clear effective communication of project status, risks, requirements.
• Develop maintain database architectures, including data lakes data warehouses.
• Ensure data quality consistency through data cleaning, transformation, validation processes.
• Lead incident analysis root cause investigations fdata-related issues, implementing improvements to enhance system stability performance.
• Evaluate possible solutions designs to establish best approach in terms of customer outcome, architecture cost. Including prototyping, technical spikes proofs of concept.
• Design, implement support scalable robust data pipelines to support analytics data processing needs.
• Implement test process automation, Test Driven Development, Continuous Integration Continuous Delivery as required support the team in implementing best practices.
• Compose high quality documentation specifications.
• Demonstrate a good understanding of the broader toolset available fdata access, analytics, manipulation etc. inder to continually evaluate/assess the suitability of tool choices.
• Work with other data scientists business teams to onboard Jupiter playbooks/python apps to robust infrastructure with appropriate standards monitoring.
• Work with Technology Governance Board to contribute towards technical standards patterns to ensure consistency adoptability across multiple service teams.
//Requirements/
• Experience: Minimum 6 years in data software engineering, with demonstrable lead 5i involvement in at least one production-grade data system within financial services a similarly regulated industry.
• Data Quality: Proven ability to validate govern data pipelines, ensuring data integrity, correctness, compliance.
• Full-Stack Engineering: Hands-on experience with Java (Spring Boot), React ( optional ), Python, covering backend, frontend, data engineering.
• Data Engineering Tools: Proficient with modern data engineering analytics platforms (e.g., Apache Airflow, Spark, Kafka, dbt, Snowflake, similar).
• DevOps & Cloud: Experience with containerisation (Docker, Kubernetes), CI/CD pipelines, cloud platforms (e.g., AWS, Azure, GCP) is highly desirable increasingly standard in the industry.
Bonus fknowledge of:
• Scripting languages, preferably Python.
• RDBMS systems, PostgresSQL, SQL server similar.
• noSQL distributed databases (MongoDB etc)
更多
