1 to 25 of 49 Remote/Hybrid Permanent PySpark Jobs

data modelling, dimensional modelling, canonical data models, and enterprise data models. Strong experience in designing ETL/ELT data pipelines. Good understanding of PySpark, Spark SQL, Python, and SQL. Strong knowledge of data warehouse, data lake, data mart, and semantic layer architecture. Experience in defining data quality, data validation ...

integration, and the software development lifecycle (SDLC). Hands-on experience with big data frameworks such as Spark, Hadoop, or Hive, including derivatives like PySpark (preferred), Spark Scala, or Spark SQL, along with familiarity with development-accelerating libraries and frameworks. Experience using data modelling tools such as ERwin ...

data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience with modern data warehousing ...

from relational databases or operational systems 2. Modern data platforms and engineering tools: Practical knowledge of data engineering tools or approaches such as Python, PySpark, Microsoft Fabric, Azure Data Factory, Synapse, Lakehouse, OneLake or equivalent platforms 3. Data quality, validation and reconciliation: Good understanding of data quality, validation, reconciliation ...

addressing any concerns and ensuring incidents are effectively managed. What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark). - A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control). - Proven ability to design data ...

OneLake, Data Factory, Synapse Data Engineering/Data Science, and Real-Time Analytics). Core Data Stack: High proficiency in advanced SQL , Python/PySpark, and data lakehouse architectures (Delta Lake). Cloud Infrastructure: Strong understanding of Microsoft Azure data services (Azure Data Factory, Azure Synapse, Azure ...

quality and security best practices Excellent stakeholder management and communication skills Experience operating within Agile delivery environments Desirable Experience Microsoft Fabric Power BI development PySpark REST API integrations Lakehouse architectures AI and Machine Learning platforms Retrieval Augmented Generation (RAG) Vector databases Generative AI solutions Power Platform technologies Data transformation ...

working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

/ELT pipelines using Azure Data Factory to ingest data from operational systems, sensors, maintenance logs, and regulatory datasets. Use Azure Databricks (SQL/PySpark) to transform, cleanse, and model large datasets for analytical consumption. Build interactive dashboards and reports in Power BI to support operational performance, asset health ...

level Performance tuning of complex SQL/T-SQL at scale Confidence taking ownership of workstreams and mentoring junior team members Desirable for both: PySpark/Python in Fabric Notebooks, Azure Data Factory/Synapse/Data Lake, Power BI, AI/ML exposure ...

data lineage. Experience managing compute across Fabric and Databricks environments. CI/CD knowledge using Azure DevOps, GitHub, and Databricks Asset Bundles. Proficiency in PySpark, Databricks SQL, T‐SQL, and Delta Lake internals. Strong understanding of Medallion Architecture and modern data modelling (Star Schema, Data Vault 2.0). Experience ...

e.g., Hadoop), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure).* Programming Skills: Programming skills in languages like Python (NumPy, pandas, PySpark), Java (Core Spark with Java, functional interface, collections), or Scala with experience in automation and scripting.* Infrastructure Tools: Experience with containerization and orchestration tools ...

years consulting (or equivalent) experience required 7+ years Databricks solution design, implementation, engineering experience required Either 5+ years building data pipelines in Databricks using PySpark, Scala and/or Spark SQL and ability to work across structured, semi-structured and unstructured data OR 5+ years ML Engineering ...

programming, concurrency, multithreading) Expertise in RESTful API design, OpenAPI/Swagger documentation and API lifecycle best practices Hands-on experience with ML libraries: PyTorch, PySpark, scikit-learn, Hugging Face Transformers Proficiency in AI tooling: Azure ML, Databricks, MLflow, LangChain, LangGraph Proven experience with Git, unit testing, containerisation and agile ...

practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence You're collaborative with excellent ...

engineers on new capabilities Opportunity to gain exposure to data modelling and machine learning projects YOUR SKILLS AND EXPERIENCE: Experience with Python and PySpark (commercial, academic, or project-based) Exposure to Databricks or similar data platforms Strong desire to learn and develop within a collaborative team Understanding of data ...

query performance Strong analytical and problem-solving skills Desirable Skills Knowledge of modern data architectures Experience with SQL, SSRS, SSIS and SSAS Experience using PySpark Familiarity with KQL Experience with data cataloguing tools Exposure to C# What you'll get in return Competitive Salary - £52,750 with a generous ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

DataBricks, Snowflake or MS Fabric. Comfortable engaging directly with stakeholders, taking ownership of technical decisions, and contributing to the broader data strategy. Essential SQL PySpark/Python DBT Azure knowledge (i.e. Logic apps) Power BI/Fabric Semantic Models Ability to work with stakeholders with their own operational pressures. ...

DataBricks, Snowflake or MS Fabric. Comfortable engaging directly with stakeholders, taking ownership of technical decisions, and contributing to the broader data strategy. Essential SQL PySpark/Python DBT Azure knowledge (i.e. Logic apps) Power BI/Fabric Semantic Models Ability to work with stakeholders with their own operational pressures. ...

Required Skills & Experience Proven experience in MLOps or DevOps roles within machine learning environments. Strong programming skills in Python, with hands‐on experience in PySpark and SQL. Deep understanding of ML lifecycle management and CI/CD best practices. Familiarity with cloud‐native ML platforms and scalable deployment strategies. ...

complex data led organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...