1、Design, build maintain batch real-time data systems pipelines
2、Develop ETL (extract, transform, load) processes to help extract manipulate data from multiple sources
3、Maintain optimize the data infrastructure required faccurate extraction, transformation, loading of data from a wide variety of data sources
Automate data workflows such as data ingestion, aggregation, ETL processing
4、Prepare raw data in OLAP databases a consumable dataset fboth technical non-technical stakeholders
5、Partner with data scientists functional leaders in different business units to deploy machine learning models
6、Ensure data accuracy, integrity, privacy, security, compliance through quality control procedures
7、Monitdata systems performance implement optimization strategies
Leverage data controls to maintain data privacy, security, compliance, quality fallocated areas of ownership
Required Knowledge Skills
1、Advanced SQL skills & experience in relational databases database design
2、Strong proficiency in data pipeline workflow management
3、Experience building deploying machine learning models
4、Experience working with Kubernetes container platform
5、Great numerical, analytical problem-solving skills
6、Excellent communicationganizational skills
7、Proven ability to work independently with a team
IT Knowledge Applications
1、Experience working with data streaming related platforms (e.g., Apache Kafka, ksqlDB, Apache Pino, etc.)
2、Experience working with large data sets distributed computing (e.g., Hadoop/Spark, Presto/Trino, Superset, etc.)
3、Proficiency in programming languages, e.g., Python, Java, Go, etc.