My Shortlist

Your shortlisted jobs will appear here. To view your shortlist, please login or register

More Jobs Like This
DATE ADDED: Tue 09/10/2018

Senior Big Data And DW Developer

Palo Alto, USA


Job Description

Job Description:

The Senior Developer is responsible for development, support, maintenance and implementation of a complex project using Infoworks Data Warehousing and ETL tools built on the Hadoop Eco System. Should have good experience in the application of standard software development principles. Should be able to work as an independent team member, capable of applying judgment to plan and execute tasks. Should be able to respond to technical queries / requests from team members and customers. Should also be able to coach, guide and mentor junior members in the team.

Job Responsibilities:

  • Position functions as a leader inside the Data Management Team to create scalable ETL and reporting solutions which meet the business needs of the organization
  • Partner with Business Analysts, Subject Matter Experts from Business Units, and counterparts in IT to complete system requirements, technical specifications and process documentation for assigned projects
  • Review, analyze, and evaluate the scope of the business problems presented and help identify viable technical solutions
  • Develop ETL and data warehousing solutions with the Infoworks product and services
  • Drive data warehouse architectural decisions and development standards
  • Create detailed technical specifications and release documentation for assigned ETL projects
  • Ensure data integrity, performance quality, and resolution of data load failures
  • Multi-task across several ongoing projects and daily duties of varying priorities as required
  • Provide input into each phase of the system development lifecycle as required
  • Ensure adherence to published development standards and hadoop best practices, resulting in consistent and efficient implementation of project components
  • Ability to make good judgments, decisions, negotiate and good problem solver

Preferred Skills:

  • HDFS, HBase, YARN, SPARK, Oozie and shell scripting
  • Background in all aspects of software engineering with strong skills in parallel data processing, data flows, REST APIs, JSON, XML, and micro service architecture.
  • At least 5 years of Design and development experience in Java/Core Java related technologies
  • At least 3 years of hands on design and development experience on Big data related technologies - Hadoop, PIG, Hive, MapReduce & Webservices
  • Strong understanding of RDBMS concepts and must have good knowledge of writing SQL and interacting with RDBMS and NoSQL database - HBase programmatically.
  • Hands on Knowledge of working with Teradata, Netezza, Oracle, DB2 databases
  • Strong understanding of File Formats - ORC, Parquet, Hadoop File formats.
  • Strong understanding and hands-on programming/scripting experience skills - UNIX shell, Python, Perl, and JavaScript.
  • Should have worked on large data sets and experience with performance tuning and troubleshooting.
  • Knowledge of Spark, Spark Streaming, Spark SQL, and Kafka is a plus.
  • Experience to Retail domain is a plus
  • Experience and desire to work in a fast paced delivery environment
  • Should be a strong communicator and be able to work independently with minimum involvement from client SMEs

Work experience:

· Minimum 8 years in IT

· Minimum 3 years with hands on experience in RDBMS technologies

· Minimum 3 plus experience in java development

· Minimum 3 years’ experience in Hadoop and NoSql solutions

· Minimum 2 years’ experience in ETL and ELT tools

· Minimum 1year experience as a lead developer

Education: Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider years of progressive experience in the specialty in lieu of Bachelor’s Degree.

Job Location:

San Francisco Bay Area

Travel Requirements:

  • Regular and expeditious travel throughout the United States is required to meet client needs and timetables
  • Available to be stationed at and work from an out-of-town client site as needed


Company Description
Infoworks is a dynamic, fast-growing, venture-funded startup that is changing the way data and information is managed. Our customers include some of the largest companies in North America. The company was founded by experienced entrepreneurs and technologists with a long history of success.

Role: Senior Big Data and DW Developer
Job Type:
Location: Palo Alto,

Apply for this job now.