Data Engineer (Azure)

Role - Data Engineer
\n Location - (Remote to Start) Dallas, USA / Philadelphia, USA

Primary Responsibilities:
\n Your responsibilities will include developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies

\n
• Design and implement Data Engineering Solutions and ETL Processes with the Azure stack including Azure Data Factory, Azure Data Lake, Azure SQL Server, Databricks, Logic Apps, Service Bus
\n
• Build data pipeline frameworks to automate high-volume batch and real-time data delivery
\n
• Design Azure data ingestion frameworks and pipelines based on the specific needs driven by the Product Owners and user stories
\n
• Continuously integrate and ship code into our cloud production environments
\n
• Work directly with Product Owners and customers to deliver data products in a collaborative and agile environment
\n
• Independently solve complex technical data engineering problems
\n
• Building data APIs and data delivery services to support critical operational and analytical applications
\n
• Contributing to the design of robust systems with an eye on the long-term maintenance and support of the application
\n
• Leveraging reusable code modules to solve problems across the team and organization
\n
• Handling multiple functions and roles for the projects and Agile teams
\n
• Defining, executing and continuously improving our internal software architecture processes
\n

Knowledge, Skills & Experience
\n Education

\n
• BS degree in Computer Science, Engineering or similar
\n
• Intermediate to senior level experience in a Data Engineering role. Demonstrated strong execution capabilities
\n

\n Required

\n
• Must have prior data engineering and ETL experience
\n
• Expert knowledge and experience with T-SQL
\n
• Hands on technology experience including Azure Data Factory, Data Lake (ADLS), Delta Lake, Databricks, Logic Apps, Service Bus
\n
• Azure Synapse, Azure stream Analytics, Azure Event hub, Azure Analysis services & CosmosDB is a plus
\n
• Significant experience with Azure SQL Server
\n
• Demonstrated experience with best Agile Scrum SDLC practices: coding standards, reviews, code management, build processes, and testing.
\n
• History of successfully developing software following an Agile methodology
\n
• Search engine integration and data catalog/metadata store experience is preferred
\n
• 5+ years of experience on designing and developing Data Pipelines for Data Ingestion or Transformation
\n
• Exposure to File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS
\n
• 5+ years of developing applications with Monitoring, Build Tools, Version Control, Unit Test, TDD, Change Management to support DevOps
\n
• At least 2 years of experience with SQL and Shell Scripting experience
\n

\n Preferred

\n
• Development Environment with Cloud Technologies
\n
• Experience with Azure DevOps CI\CD
\n
• Experience working with a combined in house and outsourced team
\n
• Experience working in a geographically separated team
\n
• 2+ years’ experience with Amazon Web Services (AWS), Microsoft Azure, Google Compute or another public cloud service
\n
• 2+ years of experience working with Streaming using Spark or Kafka or NoSQL
\n
• 2+ years of experience working with Dimensional Data Model and pipelines in relation with the same
\n
• Intermediate level experience/knowledge in at least one scripting language (Python, Scala, Pyspark)
\n
• Hands on design experience with data pipelines, joining data between structured and unstructured data
\n

\n Other

\n
• Ability to work independently
\n
• Excellent oral and written communication skills
\n
• Ability to present new ideas, approaches and information clearly
\n
• Outstanding attention to detail and organizational skills
\n
• Diligent work ethic and insatiable desire to learn and develop skills
\n
• Ability to acquire new knowledge quickly
\n
• Strong interpersonal skills
\n
• Self-starter, highly motivated
\n
• Excellent Time management skills
\n
• Cultural sensitivity/awareness
\n
• Successfully complete assessment tests offered in Pluralsight, Udemy, etc. or complete certifications to demonstrate technical expertise on more than one development platform.
\n

\n