51 to 75 of 109 PySpark Jobs

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

role focuses on designing, building and maintaining scalable data pipelines and products on a modern Databricks Lakehouse environment, using Delta Lake, Spark, Python (PySpark) and SQL to deliver high-quality, well-governed data across the business. You'll manage and optimise the cloud-based data lakehouse, leverage features like ...

reservations, Event Hub TUs, serverless compute clusters, and Terraform deployments Required Skills * Azure Databricks* Git/GitHub* DevOps & Cloud Operations* Azure Functions* Data Streaming* PySpark, Python, SQL* Automation & Public Cloud* Cloud Infrastructure* Terraform* Palantir Foundry* PowerBI* Software Lifecycle Management If you meet the requirements please send me a copy ...

Databricks-based data warehousing solution Designing and developing data models, ETL pipelines, and data integration processes Large scale data processing using PySpark Monitoring, tuning, and optimising data platforms for reliability and performance Upskilling the wider team in Databricks best practices, including modern architecture patterns Location: Atherstone (Hybrid – 3 days ...

. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

extend AI-enabled workflows as part of the core product (e.g. decision support, automation, insight generation) Develop and maintain data pipelines using Python, PySpark, SQL , and distributed systems Make pragmatic engineering decisions that balance short-term delivery with long-term product quality Contribute to technical and architectural discussions ...

World’s Most Ethical Companies for the 14th time. DUTIES (included, but not limited to): Monitor and maintain Microsoft Fabric data pipelines (Dataflows, PySpark Notebooks etc.) Develop and support self-service data transformation functionality using Powerquery (M Code) Work with front-end reporting developers to provide them with cleansed ...

Operations, DevOps, or Site Reliability Engineering managing complex systems ✅ 2+ years hands-on Cloud platform infrastructure experience in Databricks environments (Azure/AWS) ✅ Strong PySpark, Python, and Linux scripting skills in production settings ✅ Proficiency with Terraform and Ansible for provisioning and configuration management ✅ Solid experience with *nix systems (Ubuntu ...

deployed ML systems. Your Profile Essential Skills/Knowledge/Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark or PyTorch. Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer. Familiarity with cloud platforms (AWS, Azure) and ML services (e.g. ...

background in data analysis, data processing, data modeling, with excellent mathematical and problem-solving skills. A background in commercial optimisation analytics. Proficiency in Python, PySpark and SQL for data extraction, transformation, and analysis. Expertise in data-driven storage solutions and handling large, complex datasets. Good understanding of classical statistics ...

high-volume, time-series data from vehicle, sensor or IoT sources. Proven experience delivering complex ETL solutions Solid programming experience using Python, SQL and PySpark Strong stakeholder engagement skills with a diplomatic and collaborative working style Experience working within professional services or regulated environments is desirable If this role ...

strong focus on graph analytics. Proficiency in graph analytics tools and libraries such as Neo4j, NetworkX, GraphX, or similar. Strong programming skills in Python & PySpark, with experience in relevant machine learning libraries (e.g., scikit-learn, TensorFlow, PyTorch). Expertise in graph algorithms including shortest path, centrality, community detection ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

Data Engineer Data Pipelines | Python | PySpark | AWS | AWS Glue | London Location: London (hybrid - 2 days onsite per week) Salary: Up to £80K + Benefits We’re working with a tech-driven organisation looking to hire an experienced Data Engineer to help design and scale cloud-based data platforms that ...

different customer contexts and priorities Design and implement AI-enabled workflows that support decision-making and automation Build and maintain data integrations using Python, PySpark, SQL , and distributed systems Partner with customers to define success metrics and ensure solutions deliver real outcomes Feed learnings from the field back into ...

data freshness. Any AI/ML knowledge specifically around LLM's/RAG would also be useful. Data Engineer Key Skills: Python/Pyspark Databricks Delta Lake Azure LLM's/RAG Location: Remote Rate: Circa £450/Day (Outside) Contract Length: 6-12 Months Initial If this Data ...

data freshness. Any AI/ML knowledge specifically around LLM's/RAG would also be useful. Data Engineer Key Skills: Python/Pyspark Databricks Delta Lake Azure LLM's/RAG Location: Remote Rate: Circa £450/Day (Outside) Contract Length: 6-12 Months Initial If this Data ...

just which tools you used Strong SQL/SSIS/Kingswaysoft and starting to use AI and automation for ETL. Python would be useful (PySpark/Pandas/NumPy) Desirable Experience migrating data into Dynamics (or similar ERP platforms) Experience working with messy, real-world data Why Join? High ...

engineering is done across the organisation. Skills & Experience Strong hands-on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines If you're looking for a role where you can combine technical depth, ownership, and influence ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...