Brief job description
On this project, you will (along with our team) help develop a cutting edge system which generates efficient idiomatic PySpark and Pandas code for Data Engineering use cases. This code will be used to handle a wide variety of data types from multiple sources, performing transformations according to the user’s requirements. In addition to PySpark/Pandas for pure Data Engineering, the system will also have uses in generating scripts for Machine Learning / Data Science applications. Prior Machine Learning or Data Science knowledge is a plus, but not necessary to perform the role. You will use both core object oriented software engineering design principles as well as scripting techniques to delve into difficult code generation problems, finding solutions that solve important issues around readability and performance.
Must have
Pandas/PySpark
Python
PostgresSQL
Git
Docker
English language
Nice to have
AI / ML expertise
Kubernetes
Linux
AWS
Polish language
Team player
Leadership skills
Work methodology
Integration tests
Unit tests
Issue tracking tool
Knowledge repository
Code reviews
Pair programming
Version control system
Build server
Projects
We are currently looking for candidates for projects:
Working with our team of senior engineers for market leading US corporation
If you are interested, please send your resume to contact@devopsbay.io
Remember to include a personal data processing clause! I agree to the processing of personal data provided in this document for realising the recruitment process pursuant to the Personal Data Protection Act of 10 May 2018 (Journal of Laws 2018, item 1000) and in agreement with Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)
Details