20 of 20 PySpark Jobs in the South East

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

from different sources. Good communication and teamwork skills. Experience with cloud data platforms such as Databricks or Snowflake on AWS. Strong skills in Python, PySpark, and SQL. Good understanding of ETL/ELT pipelines and data transformation. Experience with data modelling and database optimisation. Strong knowledge ...

working independently and collaboratively Ability to manage priorities across multiple projects and clients Bonus Skills Experience with any of the following would be beneficial: PySpark Power BI Power Apps GCP What We’re Looking For We’re looking for someone who: Is comfortable moving between different client environments ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

Databricks Defining data models and Databricks architecture standards Establishing data governance and security frameworks Aligning data architecture with enterprise cloud strategy Supporting development using PySpark, SQL and Python within Databricks Working within CI/CD and DevOps environments (Git-based) About You: Strong technical background in Databricks and Azure ...

Looking For: Proven experience working with Palantir Foundry in production environments Strong engineering background with expertise in at least one of: Python, SQL, PySpark, or TypeScript Strong understanding of modern data architectures & distributed systems Ability to work in fast-paced, ambiguous environments Excellent stakeholder engagement and communication skills Experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

leading Professional Services firm is looking to hire a Finance Data Engineering & Analytics Lead to join their team. You will develop Python/PySpark-based data pipelines, build data models and datasets for finance, actuarial and risk reporting. You will also develop analytics, dashboards and data validation tooling. ...

data platforms and cloud environments Collaborate with engineering and architecture teams to deliver high-quality solutions Key Skills & Experience Strong experience with Python and PySpark Hands-on expertise in AWS Experience working with Databricks Ideally 5+ years working with this tech stack as a data engineer Proven background delivering ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards … reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines ...

Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/Silver/Gold data layers Embedding data quality, validation and monitoring within pipelines Supporting … Data Engineering roles Strong experience working with Azure or AWS cloud platforms Commercial, hands-on experience using Databricks in production Proven capability across: PySpark/Python/SQL Cloud data storage (ADLS or S3) ETL/ELT pipeline development Solid understanding of data architecture, modelling, and governance Experience building ...

strategies for performance and scalability Collaborate with data scientists, analysts and DevOps teams to deliver end-to-end solutions Contribute hands-on development using PySpark, SQL and Python Support CI/CD and DevOps practices using Git-based version control Present architectural designs to governance and review forums Essential … governance, metadata and security frameworks Experience working across enterprise-scale cloud data landscapes Strong understanding of performance optimisation and data management strategies Experience with PySpark, SQL and Python Spark Desirable Skills for the Databricks Solution Architect: Strong stakeholder management and communication skills Experience working within Agile delivery environments Experience ...

making across the organization. Embrace Cutting-Edge Technology: Collaborate with a tech-savvy team to leverage the latest tools and platforms, including Microsoft Fabric, PySpark, and Power BI, to build scalable, efficient, and secure data solutions. Elevate the Customer Experience: Ensure reliable data flow between core systems, empowering teams … insights to valued clients. Preferred Requirements: Proven expertise in building and operating production-ready data pipelines, with a strong background in SQL, Python, and PySpark/Spark. Proficiency in building REST API integrations, with a solid understanding of data modelling, ETL/ELT patterns, and modern data warehousing concepts. ...

Data Engineering Lead Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract till the end of the year and is required to be onsite 3 ...

Skills You’ll need to have experience with some of the following and be able to learn new technologies or concepts quickly: Python/PySpark - Experience using Python and PySpark to connect to APIs and extract data, with strong API design skills and familiarity with LLM orchestration frameworks ...

challenges Communicate findings and recommendations to stakeholders in a clear and structured manner Learning & Professional Development Build practical expertise in React, TypeScript, Python, and PySpark within a Palantir Foundry environment Apply and develop best practices in data engineering, visualisation, and business intelligence Engage with mentorship and guidance from senior … appreciation for how data can be presented in ways that are both intuitive and impactful Working knowledge of SQL, with exposure to Python or PySpark desirable but not essential Strong analytical and problem-solving skills, with a passion for extracting meaning from complex datasets Effective communication skills, both written ...

models. This isn't just a "high-level" role-we need someone who can design the blueprint and then dive into the Python/PySpark code to ensure the foundation is rock solid. Key Responsibilities: Architect & Design: Lead the implementation of on-premises, cloud, and hybrid AI solutions. GenAI … certification. AI Specialization: Proven experience with Prompt Engineering, Azure OpenAI, Document Intelligence (Form Recognizer), and Cognitive Search. Data Engineering: Expert proficiency in Python and PySpark for large-scale data processing. Operations: Familiarity with MLOps or LLMOps frameworks to support production-grade AI. Seniority: 15+ years of total IT experience ...