Job details
Python Data Engineer
Summary
Backend development in Python
Architectural design for python applications
ETL, Data Pipeline, and other DataLake-related development
Carry out business practices according to business requirements
Requirements and needs are heard from business users, and then business analysis is performed to design the optimal architecture. System development is carried out according to the architecture, and actual application development capabilities are required.
Main Project:
Improved architecture and added features to the Data Analytics system
The data analytics system of a global company collects data in various languages from various countries and regions every day, and processes it with an AI engine. In order to further expand the data to be analyzed and to improve the processing speed, it is necessary to design and develop an expansion of the pipeline or an improvement of the parallel processing architecture.
Improved ETL and data pipeline architecture and added features for DataLake
Design and implementation of ETL development, data quality monitoring, DataMart, and Catalog-related architecture for data infrastructure of multiple global companies
Personnel Requirements (Must Have)
3+ years of Python development experience
You can use SQL to extract the necessary business data.
Able to perform basic CLI operations such as ssh access and Linux commands
Be able to proactively take on new services and systems and gradually acquire knowledge and skills
The thinking skills to be able to logically analyze problems, identify causes, and come up with solutions
Japanese language level N2 or above
Personnel Requirements (Nice to Have)
Development experience using cloud environments is preferred
Experience with DataLake, ETL, GlueJob, DataFlow, etc. is preferred
Agile development experience preferred
Experience in big data analysis and development is preferred
ElasticSearch-related development and design experience preferred
Job Description
Location:
Sanai Kayabacho Building 5F, 2-13-11 Nihonbashi Kayabacho, Chuo-ku
Posted on:
9 May 2024
Fee:
Determined individually based on professional ability and experience
Treatment:
We provide a welfare system that covers all transportation expenses and provides various social insurance benefits.
Holidays:
2 days off per week (Saturday and Sunday), public holidays, year-end and New Year holidays, maternity leave, childcare leave, etc.
Working style:
・Flextime system
・Discretionary work (a work style in which the employee is free to decide how to accomplish their goals and how to allocate their time)
・Actively introduce teleworking (frequency of coming to the office varies depending on the team/project situation)