Job Details

Email to Friend Save Back

Senior Data Engineer

HOUSTON-77015, TX, US
07/02/2018

-

Required Skills

Shell, XML

Company

Infinity Consulting Solutions, Inc

Experience

Job Description

We are looking for a Big Data expert who also has a background in AWS.

Fully paid benefits with one of the best known names in the Industry!

Responsibilities:

Research, evaluate and recommend RDBMS and NoSQL products and frameworks

Design and implement data architectures according to project requirements

Design and Implement data acquisition, validation and transformation processes

Monitor and optimize performance and advise infrastructure requirements

Define data retention policies and implement appropriate archival processes

Participate in software architecture, product ideation and project planning discussions

Qualifications:

Bachelor's degree in Business, MIS, or related field

At least 10 years of software development experience in Agile and Waterfall settings

Experience with cloud (AWS preferred) based data and computing infrastructure

Extensive familiarity with Linux operating system administration

Proficient with Cassandra, Hbase, MongoDB and PostgreSQL

Proficient with developer tools such as Java, XML, JSON, REST, Shell, Perl and etc.

Proficient understanding of distributed computing principles

Experience with distributed processing using Apache Spark

Experience with various messaging systems, such as Kafka or RabbitMQ

Experience with Big Data Client toolkits, such as Mahout, SparkML, or H2O

Good understanding of Lambda Architecture, along with its advantages and drawbacks

Preferred Qualifications:

Strong understanding of SOA and microservices based applications architectures

Familiarity with business process orchestration platforms such as Pega and Appian

Experience designing advanced data analytics and machine learning applications

Experience with version control and CI/CD tooling

Upstream Oil and Gas industry experience will be a significant advantage