岗位：Python 高级架构师
主要职责
· 主导使用 Dask（重点为 dask.delayed / dask.dataframe / dask.distributed）或 PySpark 构建、优化和维护大规模分布式计算流程，设计高性能、可扩展的数据处理架构
· 对分布式计算框架进行选型、技术路线规划与执行，包括执行模型、调度策略、性能瓶颈定位与集群资源治理
· 基于业务需求（批处理、复杂依赖 DAG、实时/准实时处理等）设计端到端数据处理方案并推动落地
· 指导团队进行高质量代码开发、计算图优化、任务调度优化、容错机制设计，提升整体工程能力和可维护性
· 参与并主导核心系统的架构设计与技术方案评审，确保系统在可扩展性、稳定性和成本方面达到生产级要求
· 构建数据处理流程的可观测性体系，包括监控指标、性能分析、告警机制与容灾策略
· 与业务团队、数据团队、工程团队深度协作，将业务需求抽象为通用的计算能力、平台能力与可复用组件
· 跟踪分布式计算、Python 工程化、大数据技术生态的发展趋势，持续推进架构演进和平台能力升级
JD
1. 必备条件
· 计算机科学、软件工程、数学、统计或相关专业本科及以上学历
· 5年以上 Python 开发经验，至少 3 年以上大数据 / 分布式计算相关经验
· 深入理解 Python 底层原理、异步模型、性能分析、内存管理，熟练使用 NumPy、Pandas、Dask 等数据科学生态
· 熟悉分布式计算原理（DAG、Task Scheduler、Shuffle、Worker/Execut模型、数据分片策略等）
· 有实际的大数据处理性能调优经验，包括 CPU/内存优化、I/O 优化、序列化、并发度调优、集群资源管理等
· 具备优秀的架构设计、系统分析与解决复杂问题能力，能够独立完成大型数据处理平台的技术方案设计
· 熟悉工程化体系，如 Git、CI/CD、代码规范、自动化测试、可观测性（logging/metrics/tracing）
· 沟通能力强，能推动跨团队协作并影响技术方向
2. 优先考虑
· 在实际项目中使用 Dask（特别是 dask.delayed / Graph 优化 / distributed scheduler）的深度经验
· 有保险、再保险、金融、风控等行业的大规模数据处理经验
· 熟悉其他分布式计算框架（Spark、Ray、Flink、阿里 MaxCompute、AWS EMR 等）
· 熟悉任务编排和数据工作流工具（Airflow、Prefect、Dagster 等）
· 熟悉云平台（AWS / Azure / GCP），尤其是分布式存储、Serverless、K8s Operator、集群自动伸缩等
· 有实时计算经验（如 Flink、Spark Structured Streaming、Kafka Streams）、低延迟管道调优经验
· 有数据平台建设经验，如数据质量体系、血缘管理、数据治理、统一指标体系
· 具备 Dash / Streamlit / Superset / Tableau 等数据可视化开发经验
· 有技术分享、开源贡献或架构方案沉淀经验者优先
Position: SeniPython Architect
Key Responsibilities
· Lead the design, optimization, maintenance of large-scale distributed computing workflows using Dask (especially dask.delayed, dask.dataframe, dask.distributed) PySpark, architect high-performance, scalable data processing systems
· Drive framework selection, architectural planning, technical roadmap execution fdistributed computing, including execution models, scheduling strategies, performance bottleneck analysis, cluster resource governance
· Design end-to-end data processing solutions based on business requirements (batch processing, complex DAG dependencies, real-time / near–real-time processing) ensure successful production implementation
· Guide the team in writing high-quality, maintainable code; optimize computation graphs, scheduling strategies, fault tolerance mechanisms; elevate overall engineering standards
· Lead participate in architecture reviews technical design discussions, ensuring system scalability, stability, cost efficiency at production scale
· Build robust observability fdata processing workflows, including monitoring metrics, performance analysis, alerting mechanisms, disaster recovery strategies
· Collaborate closely with business, data, engineering teams to translate domain needs reusable computation capabilities platform components
· Stay current with trends in distributed computing, Python engineering, big data ecosystems, continuously driving architectural evolution platform upgrades
Job Requirements
1. Basic Qualifications
· Bachelor’s degree above in Computer Science, Software Engineering, Mathematics, Statistics, related fields
· 5+ years of Python development experience, with at least 3 years in big data distributed computing
· Deep understanding of Python internals, asynchronous models, performance profiling, memory management; proficient with NumPy, Pandas, Dask, related data science ecosystems
· Strong understanding of distributed computing fundamentals (DAGs, task schedulers, shuffle mechanisms, worker/executmodels, data partitioning strategies, etc.)
· Proven experience optimizing large-scale data processing systems, including CPU/memory tuning, I/O optimization, serialization, parallelism tuning, cluster resource management
· Strong architectural design skills with the ability to independently design technical solutions flarge-scale data processing platforms
· Familiar with engineering best practices such as Git, CI/CD, coding standards, automated testing, observability (logging/metrics/tracing)
· Excellent communication cross-team collaboration skills, with the ability to influence technical direction
2. Preferred Qualifications
· Hands-on experience with Dask in production environments, especially in dask.delayed, computation graph optimization, distributed scheduler tuning
· Experience with data processing in insurance, reinsurance, financial services, risk modeling, similar domains
· Familiar with other distributed computing frameworks such as Spark, Ray, Flink, MaxCompute, AWS EMR
· Experience with workflowchestration tools such as Airflow, Prefect, Dagster
· Strong knowledge of cloud platforms (AWS / Azure / GCP), including distributed storage, serverless computing, Kubernetes operators, autoscaling strategies
· Experience in real-time computing frameworks such as Flink, Spark Structured Streaming, Kafka Streams, with low-latency pipeline tuning
· Experience building data platforms, including data quality frameworks, lineage tracking, data governance, metric unification
· Experience with data visualization tools such as Dash, Streamlit, Superset, Tableau
· Priexperience in technical sharing, open-source contributions, architectural documentation is a strong plus
更多
