Job responsibilities

1. Bachelor degree or above, major in computer science and related fields, more than 5 years of work experience in big data development, data warehouse construction or related fields;

2. Have a global business perspective and platform thinking, be able to build basic data models and ontologies for all business scenarios, and establish relationships between data based on the actual business situation.

3. Rich experience in data lake/data warehouse construction, familiar with data aggregation, cleaning, transformation, and loading (ETL) processes, and able to efficiently design and implement data processing architectures.

4. Proficient in at least one mainstream big data processing technology stack, such as Hadoop, Spark, Flink, Hive, Presto, etc., and be able to conduct in-depth development and optimization based on these technologies;

5. Familiar with data warehouse modeling and design, able to design efficient and scalable data models according to business needs, supporting offline, real-time, and AI application scenarios;

6. Experience in optimizing data processing performance, able to tune for large-scale data processing tasks to ensure high concurrency, high availability and high reliability of data processing;

7. Familiar with the basic concepts of machine learning and deep learning, understand the basic process of large model training and fine-tuning, and be able to provide data support and optimization suggestions for algorithm engineers.

8. Proficient in at least one programming language (Java, Python, Go, etc.), with good programming habits and code specification awareness, and able to write high-quality and maintainable code;

9. Familiar with Linux operating system, with a solid foundation in data structure and algorithms, and able to efficiently handle complex data problems.

Qualifications

1. Responsible for the infrastructure construction of the company's internal data lake/data warehouse, create an efficient and reliable data aggregation and cleaning platform, and provide high-quality data support for artificial intelligence applications. Design and implement data aggregation, cleaning, transformation, and loading (ETL) processes to ensure data accuracy, integrity, and consistency to support the company's internal business needs and data analysis efforts.

2. Data ontology construction based on all business scenarios;

3. Optimize the data processing process and architecture, and carry out in-depth development and optimization based on the company's internal offline, real-time and AI application scenarios to improve data processing efficiency and performance.

4. Support large model training and fine-tuning, work closely with algorithm engineers, provide data support and optimization suggestions, and ensure that data can be efficiently used for model training and fine-tuning;

5. Write data development documents, record data processing processes, data architecture design and technical implementation details, and provide technical guidance and reference for team members.

6. Assist the business department to process massive data, solve difficult problems in the process of data processing, give full play to the value of data, and provide support for business decision-making.

7. Continuously optimize the performance and stability of the data platform to ensure high concurrency, high availability and high reliability of data processing.