Cloud Data Engineer

Five Reasons to Work with Us:  

1. Work with a leader in financial services industry  
2. Supportive & flexible remote or hybrid environment 
3. Diverse & friendly culture 
4. Fast paced growing company 
5. A company that cares with community service efforts

What You Will Be Doing: 

The Cloud Data Engineer will design, build, and operate data pipelines that power both analytics and AI-driven product capabilities at RegEd — a fintech platform serving the compliance and regulatory needs of the financial services industry. Working within a Data Engineering team embedded in a broader 60-person engineering organization, you'll own the full lifecycle of ETL/ELT workflows on Azure, integrate with AI and ML systems, and help define standards that scale across teams.

This role requires a service-oriented mentality and high sense of ownership. The Cloud Data Engineer would be expected to continually look for areas of improvement in projects, products and processes to ensure efficiency, consistency and accuracy. Self-direction and the ability to manage and allocate your time over multiple simultaneous projects are keys for success.

Responsibilities:

• Design and implement scalable ETL/ELT pipelines using Azure Data Factory, Databricks, and related Azure data services.
• Build and maintain data ingestion workflows that feed ML model training, inference, and AI feature stores.
• Develop reusable pipeline components and data transformation patterns using PySpark and/or SQL in Databricks notebooks and jobs.
• Collaborate with data architects and ML engineers to support AI initiatives — including embedding pipelines, vector store population, and LLM-ready data preparation.
• Monitor pipeline performance, diagnose failures, and drive improvements in reliability and throughput.
• Contribute to schema design, data modeling, and metadata management in Unity Catalog or Azure Purview.
• Participate in code reviews and help establish engineering standards for the data platform team.
• Document pipeline architecture, data lineage, and operational runbooks.

Must Have: 

• Bachelor's Degree or equivalent work experience.
• 3–5 years building production data pipelines in a cloud-native environment.
• Hands-on experience with Azure Data Factory (ADF) — authoring pipelines, linked services, triggers, and monitoring. 
• Proficiency with Azure Databricks — PySpark, Delta Lake, job clusters, and notebook-based workflows. 
• Strong SQL skills across analytical and transactional workloads.
• Experience integrating pipelines with AI or ML workflows — data prep for model training, feature engineering, or model output ingestion. 
• Understanding of ELT design patterns, idempotency, and pipeline observability. 
• Familiarity with Git-based CI/CD workflows for data pipeline deployment.

Nice To Have: 

• Knowledge of the financial services / insurance / securities compliance industry’s concepts, practices, and procedures a plus. 
• Masters' Degree in a related field.

The Process: 

We can hire in the following states: Alabama, Arizona, California, Colorado, Connecticut, Delaware, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Kentucky, Maryland, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, New Jersey, New York, North Carolina, Ohio, Pennsylvania, South Carolina, Tennessee, Texas, Utah, and Wisconsin.