Data Engineer III - Python, Pyspark, Databricks
JPMorganChase
This job is no longer accepting applications
See open jobs at JPMorganChase.See open jobs similar to "Data Engineer III - Python, Pyspark, Databricks" Italy-America Chamber of Commerce.Job Description
Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team.
The Cash Account Platform (CAP) is a firm-wide, strategic initiative to enhance the risk management, oversight, control and reporting related to JP Morgan’s intraday liquidity. In support of this the platform provides real-time dashboards for senior executives which visualize Cash, Credit, and Collateral balances & activity at the Firmwide, Line of Business and Institutional Client level. It is also responsible for the firm’s cash forecast and funding functions, in conjunction with the nostro account reference data system. The Bournemouth development team is one of several global Agile teams contributing to the active development of the suite of applications – with emphasis in the business areas pertaining to Cash Management..
Job responsibilities
- Develop and Maintain Data Pipelines:
- Design, build, and maintain efficient, reusable, and reliable data pipelines.
- Ensure the robustness and reliability of data processing systems.
- Data Integration:
- Integrate data from various sources, including databases, APIs, and flat files.
- Implement ETL processes to support data transformation and loading.
- Data Migration:
- Contribute to the migration of data from RDMS to cloud data source.
- Design and implement robust data models to support analytical use cases.
- Database Management:
- Work with relational and NoSQL databases to store and retrieve data.
- Optimize database performance and ensure data integrity.
- Collaborate with Data Teams:
- Work closely with data analysts, data scientists, and other stakeholders to understand data requirements and deliver solutions.
- Participate in code reviews and provide constructive feedback.
- Implement Data Strategy:
- Contribute to the development and execution of the data strategy.
- Assist in the design and implementation of data governance and data quality frameworks.
- Automation and Scripting:
- Develop scripts to automate repetitive tasks and improve data processing efficiency.
- Ensure scripts are well-documented and maintainable.
- Performance Tuning:
- Identify bottlenecks and bugs, and devise solutions to address these issues.
- Optimize the performance of data processing workflows.
- Documentation:
- Maintain comprehensive documentation for all data processes, pipelines, and systems.
- Ensure that documentation is up-to-date and accessible to relevant stakeholders.
Required qualifications, capabilities, and skills
- Proficient at Python and relevant libraries (e.g., Pandas, NumPy, SQLAlchemy).
- Expert in Database, PL\SQL, Performance tuning, DB modelling, Erwin, DB query review, database query optimization.
- Experienced development in a data lake area using Databricks, Redshift or Snowflake tools.
- Experience of working on streaming data applications such as Spark Streaming, Kafka, MSK, Kinesis.
- Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and their data services.
- Knowledge of machine learning and data science principles.
- Understanding of data governance and data quality principles..
- Experience working with Tableau,
- Experience with DevOps including Continuous Integration (CI) and Continuous Deployment (CD) tools e.g. Jenkins, Sonar.
- Exposure to scheduling tools like Autosys / Control-M.
- Experience with big data technologies (e.g. Spark, Hadoop).
- Familiarity with data integration tools (e.g., Apache Airflow) and data warehousing solutions (e.g.Databricks, Redshift).
- Work effectively within Agile development framework to ensure timely and efficient project delivery
This job is no longer accepting applications
See open jobs at JPMorganChase.See open jobs similar to "Data Engineer III - Python, Pyspark, Databricks" Italy-America Chamber of Commerce.