Job Description
Job Title: Data Engineer Scope of Services The selected candidate will be responsible for:
- ssessing feasibility and technical requirements for LINKS → DataLake integration.
- Collaborating with OPH Immunization Program, OPH Bureau of Health Informatics, and STChealth on data specifications and recurring ingestion pipelines.
- Building and optimizing ETL workflows for LINKS and complementary datasets (e.g., Vital Records, labs, registries).
- Designing scalable data workflows to enhance data quality, integrity, and identity resolution.
- Implementing data governance, observability, and lineage tracking across all pipelines.
- Mentoring engineers, supporting testing activities, and enforcing best practices in orchestration and architecture.
- Documenting and communicating technical solutions to both technical and non-technical stakeholders.
Mandatory Expertise & Experience - 3 years of experience in data engineering and/or data architecture.
- 2 years of experience with Python for ETL and automation (pandas, requests, API integration).
- 2 years of hands-on experience with SQL queries, stored procedures, and performance tuning (Oracle, SQL Server, MySQL preferred).
- 1 year of experience with ETL orchestration tools (e.g., Prefect, Airflow, or equivalent).
- 1 year of experience with cloud platforms (Azure, AWS, or GCP), including data onboarding/migration.
- 1 year of exposure to data lake / medallion architecture (bronze, silver, gold).
- 2 years of experience preparing written documentation and conducting verbal communication for cross-functional collaboration.
Preferred (Not Mandatory) Expertise & Experience - 5+ years of data engineering experience.
- Experience integrating or developing REST/JSON or XML APIs.
- Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps, etc.).
- Exposure to Infrastructure as Code (Terraform, CloudFormation).
- Experience with data governance and metadata tools (Atlan, OpenMetadata, Collibra).
- Experience with public health or healthcare datasets, including PHI/PII handling.
- Familiarity with SAS and R workflows for supporting epidemiologists and analysts.
- Experience with additional SQL platforms (Postgres, Snowflake, Redshift, BigQuery).
- Familiarity with data quality frameworks (Great Expectations, Deequ).
- Experience with real-time/streaming tools (Kafka, Spark Streaming).
- Familiarity with big data frameworks for large-scale transformations (Spark, Hadoop).
- Knowledge of data security and compliance frameworks (HIPAA, SOC 2, etc.).
- Experience working in an Agile/Scrum environment.
***Lumen and / or its clients will not provide equipment (Laptop, monitor, etc.) to the selected contractor. The contractor must have their own equipment. Access to a virtual desktop set up (software) will be provided by Lumen's client, allowing the user access to the required systems and technology.***
Lumen Solutions Group Inc. is an Equal Opportunity Employer and does not discriminate in employment on the basis of Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.
Job Tags
For contractors, Work at office, Remote work