My Shortlist

Your shortlisted jobs will appear here. To view your shortlist, please login or register

More Jobs Like This
DATE ADDED: Thu 31/10/2019

Sr.Data Engineer

San Francisco, CA, US


JOB TYPE: Contract, FullTime

Sr.Data Engineer

San Francisco,CA

12 Months Contract

Job Description

Assist migration of existing data aggregation ETL techniques to new data pipeline

Operationalize streaming data flows and setting up streaming data processors

Set up pipeline tools for scalability and monitoring

Collaborate with data scientists in prototyping and operationalizing machine learning models qualifications:

Experience with setting up data pipelines with Apache Kafka and Kafka Connect

Experience with Java and/or Python

Running and optimizing SQL queries on RDBMS like MS SQL server, MySQL/MariaDB (at least one NoSQL database like MongoDB, Cassandra or HBase)

Knowledge of Enterprise Data Architectures, Distributed and Microservice Software Architectures and Design Patterns

Working knowledge of producing and consuming REST services

Working knowledge of CI/CD tools like Jenkins, UCD or Ansible, Git repositories like GitHub or BitBucket

BI reporting using Tableau and working knowledge of Bash scripting Nice to have

Experience with ELK stack (Logstash, ElasticSearch), and / or Container-based Virtualization (Kubernetes, Docker)

Experience with MapReduce, Hive or Spark

- provided by Dice