PySpark Developers
- USM Business System
248 Days Ago
NA
NA
Mclean-VA, Richmond-VA
7-9 Years
Job Description
Role: PySpark Developers
- Locations: McLean, VA or Richmond, VA
- Openings: 3 positions
-Rate : Depends Exp
-Client: USM Business System
- Interview Process: One round
Requirements
- Experience: Ideally 5 years, but flexible
- Skills:
- Proficiency in PySpark for data engineering tasks
- Experience with AWS, particularly EMR and Hadoop
- Strong ability to write and maintain PySpark pipelines
- Knowledge of test automation, including writing unit tests (99.5% test coverage required)
- Experience with regulatory reporting systems is a plus
- Databricks experience is nice to have
Responsibilities
- Develop and manage PySpark pipelines for large-scale regulatory reporting applications.
- Handle data transformations and integrate data from various systems across Capital One.
- Ensure high code quality with robust unit tests.
- Work on a large-scale application that handles multiple regulatory reporting use cases, which for other banks would require separate applications.
