Szczegóły oferty pracy

Big Data Developer

Aplikuj

Lokalizacja: Zdalnie Dodano: 2024-01-08

Company providing innovative trading, currency data and analytics solutions

Understand the data sources and map them into respective domains
Build projects using complex business algorithms and load them in the data warehouse.
Present a schema management system design at an Engineering-wide architecture review
Build new data pipelines using DataFlow and DataForm
Code data fills in python against Google Cloud Storage and BigQuery
Investigate, deploy and implement a caching solution for enabling real-time joins of big data sets
Develop a data quality validation framework used for both testing and real time production data
Design tools enabling easy-to-use workflows for internal teams using the data platform

7+ years of experience designing and implementing large scale software
4+ years of experience working with Big Data technologies like Apache
Beam,DataFlow, BigQuery, Kafka, Airflow and Spark
Experience designing a real-time Big Data platform
Experience deploying and managing Big Data infrastructure
Strong coding ability in an object oriented language (preferably Scala, Python, Java)
Excellent team player with strong communication skills (verbal and written)
Enthusiastic about collaborative problem solving
Bachelor’s degree or better in Computer Science

Nice to have:

Experience with Streamsets, Google BigQuery, Apache NiFi, Looker, Pipeline monitoring tools
Strong SQL and data modelling skills