Data Engineer

Benefits Data Trust (BDT) seeks a Data Engineer. As part of our Data Science team, the Data Engineer will research, develop, deploy, and support the underlying data-flow and the data warehousing solutions to ensure scale and reliability. This is a collaborative role that requires working with our cloud architecture, platform development, and machine learning teams.

The Data Engineer will be responsible for building our enterprise data lake and helping to create data marts for a diverse set of business use cases. The Data Engineer will organize, categorize, store, and transform data from both internal systems
and external partners. In partnership with our cloud engineering, product development, and projects teams, the Data Engineer is responsible for automating and monitoring our data pipelines that assist the organization to better support our clients in their pursuit of healthier, more independent lives.Based in our Center City Philadelphia office, the Data Engineerreports to the Senior Data Scientist.

 

Responsibilities

  • Design scalable, automated data pipelines using both managed and unmanaged cloud services
  • Enhance data collection procedures to include information that is relevant for building analytic systems
  • Create our enterprise data lake
  • Work with stakeholders to design SLAs for pipeline solutions and data stores
  • Monitor and review existing solutions and propose strategic improvements
  • Stay up to date on the latest technology developments and offerings to keep the team current

Qualifications

  • Demonstrated experience with data warehouses or data lakes and relational databases using managed cloud service (e.g., GCP, AWS)
  • Experience programming with Python and Ruby
  • Prior experience with data pipeline tools such as Airflow, Luigi and NiFi
  • Advanced level of knowledge of SQL, including query optimization, is required
  • Experience with serverless and micro-services infrastructure, knowledge of cloud functions and working with languages like Go
  • Familiarity with messaging systems (queues, publish-subscribe architecture) is desired, bonus points for experience with Kafka, Rabbit, AWS SQS or Google PubSub
  • Ability to learn emerging technologies and integrate them into production level systemsPrevious experience building data-pipelines and ETL solutions

About BDT

Benefits Data Trust (BDT)is a national nonprofit that helps people live healthier, more independent lives by creating smarter ways to access essential benefits and services. Each year, BDT helps tens of thousands of people receive critical supports using data, technology, targeted outreach, and policy change. Since inception in 2005, BDT has submitted over 800,000 applications – more than any other single entity in the country – securing over $7 billion in benefits and services. BDT employs more than 170 people and provides enrollment assistance to individuals in six states, and policy assistance to states nationwide. For more information, visit bdtrust.org.

To apply for this position, click here.